[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:51:17 dist-test-slave-jlzn krb5kdc[10638](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:51:17 dist-test-slave-jlzn krb5kdc[10638](info): set up 2 sockets
Aug 10 00:51:17 dist-test-slave-jlzn krb5kdc[10638](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 00:51:18.769279 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.006s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:51:18 dist-test-slave-jlzn krb5kdc[10638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:51:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:51:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:51:18.937875 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:41205
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:37439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:41205
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:19.280476 10660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:19.280879 10660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:19.280977 10660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:19.287819 10660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:51:19.287914 10660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:19.287966 10660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:19.288021 10660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:51:19.288064 10660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:51:19.297534 10660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:37439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:41205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:41205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:19.299191 10660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:19.301137 10660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:19.319095 10666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:19.319577 10670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:19.319728 10667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:19.320405 10660 server_base.cc:1034] running on GCE node
I20260810 00:51:19.321425 10660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:19.323483 10660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:19.324729 10660 hybrid_clock.cc:648] HybridClock initialized: now 1786323079324591 us; error 123 us; skew 500 ppm
I20260810 00:51:19.325238 10660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:19 dist-test-slave-jlzn krb5kdc[10638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323079, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:19.331295 10660 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:51:19.333482 10660 webserver.cc:460] Webserver started at http://127.10.97.126:33373/ using document root <none> and password file <none>
I20260810 00:51:19.334481 10660 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:19.334606 10660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:19.335053 10660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:19.338971 10660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "021d2cd03e0d4aee8e207d3b8689666a"
format_stamp: "Formatted at 2026-08-10 00:51:19 on dist-test-slave-jlzn"
server_key: "427311a61ba0db1ed455482688afa0c4"
server_key_iv: "2733bda72a5a460ffb0a276c80998b7e"
server_key_version: "encryptionkey@0"
I20260810 00:51:19.339757 10660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "021d2cd03e0d4aee8e207d3b8689666a"
format_stamp: "Formatted at 2026-08-10 00:51:19 on dist-test-slave-jlzn"
server_key: "427311a61ba0db1ed455482688afa0c4"
server_key_iv: "2733bda72a5a460ffb0a276c80998b7e"
server_key_version: "encryptionkey@0"
I20260810 00:51:19.347149 10660 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.004s
I20260810 00:51:19.352490 10681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:19.354285 10660 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 00:51:19.354539 10660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "021d2cd03e0d4aee8e207d3b8689666a"
format_stamp: "Formatted at 2026-08-10 00:51:19 on dist-test-slave-jlzn"
server_key: "427311a61ba0db1ed455482688afa0c4"
server_key_iv: "2733bda72a5a460ffb0a276c80998b7e"
server_key_version: "encryptionkey@0"
I20260810 00:51:19.354753 10660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:19.382949 10660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:19.387437 10660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:19.387755 10660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:19.408370 10660 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:41205
I20260810 00:51:19.408408 10770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:41205 every 8 connection(s)
I20260810 00:51:19.410524 10660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:51:19.412066 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 10660
I20260810 00:51:19.412416 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:51:19.414512 10629 external_mini_cluster.cc:1428] Setting key 68593b8c318af134fe7f620ca2858aee
I20260810 00:51:19.417268 10771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:19.432430 10771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a: Bootstrap starting.
I20260810 00:51:19.438891 10771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:19.440608 10771 log.cc:824] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:19.444206 10771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a: No bootstrap required, opened a new log
Aug 10 00:51:19 dist-test-slave-jlzn krb5kdc[10638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:19.453501 10771 raft_consensus.cc:348] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021d2cd03e0d4aee8e207d3b8689666a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41205 } }
I20260810 00:51:19.454077 10771 raft_consensus.cc:374] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:19.454263 10771 raft_consensus.cc:729] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 021d2cd03e0d4aee8e207d3b8689666a, State: Initialized, Role: FOLLOWER
I20260810 00:51:19.455300 10771 consensus_queue.cc:260] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021d2cd03e0d4aee8e207d3b8689666a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41205 } }
I20260810 00:51:19.455675 10771 raft_consensus.cc:388] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:51:19.455986 10771 raft_consensus.cc:482] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:51:19.456240 10771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:19.456799 10774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:19.419997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57568 (local address 127.10.97.126:41205)
0810 00:51:19.421097 (+ 1100us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:19.421113 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:19.423604 (+ 2491us) server_negotiation.cc:407] Connection header received
0810 00:51:19.433433 (+ 9829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:19.433460 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:19.434102 (+ 642us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:19.434570 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:19.437400 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:19.439429 (+ 2029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:19.441062 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:19.441868 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:19.442109 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:19.448018 (+ 5909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:19.448085 (+ 67us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:19.448122 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:19.448164 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:19.451178 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:19.451989 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:19.451996 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:19.452001 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:19.452219 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:19.452671 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:19.452675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:19.452677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:19.453284 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:19.453689 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:19.454853 (+ 1164us) server_negotiation.cc:299] Negotiation successful
0810 00:51:19.455242 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":380,"threads_started":1}
I20260810 00:51:19.458765 10771 raft_consensus.cc:504] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021d2cd03e0d4aee8e207d3b8689666a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41205 } }
I20260810 00:51:19.459739 10771 leader_election.cc:302] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 021d2cd03e0d4aee8e207d3b8689666a; no voters:
I20260810 00:51:19.460561 10771 leader_election.cc:288] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:51:19.460822 10778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:19.462168 10778 raft_consensus.cc:686] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a [term 1 LEADER]: Becoming Leader. State: Replica: 021d2cd03e0d4aee8e207d3b8689666a, State: Running, Role: LEADER
I20260810 00:51:19.462954 10778 consensus_queue.cc:237] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021d2cd03e0d4aee8e207d3b8689666a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41205 } }
I20260810 00:51:19.463608 10771 sys_catalog.cc:564] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:51:19.467777 10781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 021d2cd03e0d4aee8e207d3b8689666a. Latest consensus state: current_term: 1 leader_uuid: "021d2cd03e0d4aee8e207d3b8689666a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021d2cd03e0d4aee8e207d3b8689666a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41205 } } }
I20260810 00:51:19.468219 10780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "021d2cd03e0d4aee8e207d3b8689666a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021d2cd03e0d4aee8e207d3b8689666a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41205 } } }
I20260810 00:51:19.468227 10781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a [sys.catalog]: This master's current role is: LEADER
I20260810 00:51:19.468542 10780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a [sys.catalog]: This master's current role is: LEADER
I20260810 00:51:19.470342 10788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:51:19.478101 10788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:51:19.491117 10788 catalog_manager.cc:1347] Generated new cluster ID: c59d0de4f2b94d5898d445af471fd26d
I20260810 00:51:19.491221 10788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:51:19.519035 10788 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:51:19.520730 10788 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:51:19.526947 10788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 021d2cd03e0d4aee8e207d3b8689666a: Generated new TSK 0
I20260810 00:51:19.527796 10788 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
I20260810 00:51:19.582464 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:41205
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:37439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:19.906937 10811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:19.907177 10811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:19.907284 10811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:19.914204 10811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:19.914300 10811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:19.914407 10811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:51:19.923472 10811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:37439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:41205
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:19.924535 10811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:19.925701 10811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:19.938478 10816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:19.938488 10817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:19.938706 10821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:19.939251 10811 server_base.cc:1034] running on GCE node
I20260810 00:51:19.939791 10811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:19.940634 10811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:19.941838 10811 hybrid_clock.cc:648] HybridClock initialized: now 1786323079941795 us; error 31 us; skew 500 ppm
I20260810 00:51:19.942140 10811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:19 dist-test-slave-jlzn krb5kdc[10638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323079, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:19.945173 10811 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:51:19.946480 10811 webserver.cc:460] Webserver started at http://127.10.97.65:38041/ using document root <none> and password file <none>
I20260810 00:51:19.947115 10811 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:19.947247 10811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:19.947549 10811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:19.950260 10811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "7dd9d5a67b60403ab3d579c1fba09546"
format_stamp: "Formatted at 2026-08-10 00:51:19 on dist-test-slave-jlzn"
server_key: "32d8b6fd903ca9212dd1a426ca85d3cd"
server_key_iv: "43bb02f0d223399802e4d64b0a4ee286"
server_key_version: "encryptionkey@0"
I20260810 00:51:19.950984 10811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "7dd9d5a67b60403ab3d579c1fba09546"
format_stamp: "Formatted at 2026-08-10 00:51:19 on dist-test-slave-jlzn"
server_key: "32d8b6fd903ca9212dd1a426ca85d3cd"
server_key_iv: "43bb02f0d223399802e4d64b0a4ee286"
server_key_version: "encryptionkey@0"
I20260810 00:51:19.955703 10811 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 00:51:19.959272 10830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:19.960618 10811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 00:51:19.960773 10811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "7dd9d5a67b60403ab3d579c1fba09546"
format_stamp: "Formatted at 2026-08-10 00:51:19 on dist-test-slave-jlzn"
server_key: "32d8b6fd903ca9212dd1a426ca85d3cd"
server_key_iv: "43bb02f0d223399802e4d64b0a4ee286"
server_key_version: "encryptionkey@0"
I20260810 00:51:19.960974 10811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:19.989984 10811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:19.993824 10811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:19.994098 10811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:19.995081 10811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:19.996680 10811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:19.996758 10811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:19.996937 10811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:19.997016 10811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:20.027837 10811 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:33341
I20260810 00:51:20.027908 10993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:33341 every 8 connection(s)
I20260810 00:51:20.028980 10811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:51:20.038559 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 10811
I20260810 00:51:20.038771 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:51:20.039535 10629 external_mini_cluster.cc:1428] Setting key 18f29cd7ba16830b07fb8e0ce0aff9e7
Aug 10 00:51:20 dist-test-slave-jlzn krb5kdc[10638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323079, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:20.048559 10998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:20.031945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:58465 (local address 127.10.97.126:41205)
0810 00:51:20.032347 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:20.032351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:20.033185 (+ 834us) server_negotiation.cc:407] Connection header received
0810 00:51:20.034932 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:20.034938 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:20.035017 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:20.035113 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:20.036555 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:20.037260 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:20.038794 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:20.039408 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:20.039544 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:20.042923 (+ 3379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:20.042931 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:20.042936 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:20.042974 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:20.045864 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:20.046502 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:20.046507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:20.046510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:20.046661 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:20.047027 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:20.047031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:20.047033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:20.047329 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:20.047461 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:20.048163 (+ 702us) server_negotiation.cc:299] Negotiation successful
0810 00:51:20.048317 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":213,"threads_started":1}
I20260810 00:51:20.049738 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:20.032380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:41205 (local address 127.10.97.65:58465)
0810 00:51:20.033042 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:20.033078 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:20.034679 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:20.035346 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:20.035354 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:20.035844 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:20.036381 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:20.036398 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:20.037409 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:20.037412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:20.038641 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:20.038653 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:20.039617 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:20.039625 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:20.039867 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:20.040453 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:20.040474 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:20.042707 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:20.046000 (+ 3293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:20.046006 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:20.046017 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:20.046384 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:20.046747 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:20.046750 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:20.046752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:20.046927 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:20.047425 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:20.047430 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:20.047705 (+ 275us) client_negotiation.cc:769] Sending connection context
0810 00:51:20.048069 (+ 364us) client_negotiation.cc:240] Negotiation successful
0810 00:51:20.048538 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":272,"threads_started":1}
I20260810 00:51:20.051167 10994 heartbeater.cc:347] Connected to a master server at 127.10.97.126:41205
I20260810 00:51:20.051461 10994 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:20.052392 10994 heartbeater.cc:511] Master 127.10.97.126:41205 requested a full tablet report, sending...
I20260810 00:51:20.056519 10706 ts_manager.cc:194] Registered new tserver with Master: 7dd9d5a67b60403ab3d579c1fba09546 (127.10.97.65:33341)
WARNING: no policy specified for kudu/127.10.97.66@KRBTEST.COM; defaulting to no policy
I20260810 00:51:20.059384 10706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:58465
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
I20260810 00:51:20.080883 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:41205
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:37439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:20.389737 11005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:20.389986 11005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:20.390123 11005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:20.397034 11005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:20.397138 11005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:20.397282 11005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:51:20.410709 11005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:37439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:41205
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:20.412328 11005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:20.414340 11005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:20.434680 11012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:20.435655 11017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:20.435851 11020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:20.436889 11005 server_base.cc:1034] running on GCE node
I20260810 00:51:20.437597 11005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:20.438726 11005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:20.439985 11005 hybrid_clock.cc:648] HybridClock initialized: now 1786323080439851 us; error 34 us; skew 500 ppm
I20260810 00:51:20.440476 11005 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:20 dist-test-slave-jlzn krb5kdc[10638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323080, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:20.445022 11005 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:51:20.446560 11005 webserver.cc:460] Webserver started at http://127.10.97.66:34843/ using document root <none> and password file <none>
I20260810 00:51:20.447386 11005 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:20.447559 11005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:20.447952 11005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:20.451356 11005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "e5e9aebf12624083a5d36d406554f237"
format_stamp: "Formatted at 2026-08-10 00:51:20 on dist-test-slave-jlzn"
server_key: "7ae6b0b64026fc6f0b2b27408e74febc"
server_key_iv: "ddc601ed5a444943cd65db19ed22e295"
server_key_version: "encryptionkey@0"
I20260810 00:51:20.452265 11005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "e5e9aebf12624083a5d36d406554f237"
format_stamp: "Formatted at 2026-08-10 00:51:20 on dist-test-slave-jlzn"
server_key: "7ae6b0b64026fc6f0b2b27408e74febc"
server_key_iv: "ddc601ed5a444943cd65db19ed22e295"
server_key_version: "encryptionkey@0"
I20260810 00:51:20.458333 11005 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 00:51:20.461804 11030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:20.463008 11005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 00:51:20.463150 11005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "e5e9aebf12624083a5d36d406554f237"
format_stamp: "Formatted at 2026-08-10 00:51:20 on dist-test-slave-jlzn"
server_key: "7ae6b0b64026fc6f0b2b27408e74febc"
server_key_iv: "ddc601ed5a444943cd65db19ed22e295"
server_key_version: "encryptionkey@0"
I20260810 00:51:20.463338 11005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:20.511991 11005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:20.515836 11005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:20.516090 11005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:20.517138 11005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:20.518369 11005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:20.518448 11005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:20.518569 11005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:20.518646 11005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:20.550347 11005 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:34375
I20260810 00:51:20.550422 11196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:34375 every 8 connection(s)
I20260810 00:51:20.551604 11005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:51:20.553794 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 11005
I20260810 00:51:20.553975 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:51:20.554423 10629 external_mini_cluster.cc:1428] Setting key 50cc9a9c6a0cd64521010d6aa45ed496
Aug 10 00:51:20 dist-test-slave-jlzn krb5kdc[10638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323080, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:20.571100 11199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:20.554030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:49825 (local address 127.10.97.126:41205)
0810 00:51:20.554494 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:20.554500 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:20.555447 (+ 947us) server_negotiation.cc:407] Connection header received
0810 00:51:20.557333 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:20.557339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:20.557414 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:20.557519 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:20.559087 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:20.559893 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:20.561392 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:20.561845 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:20.561957 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:20.565540 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:20.565546 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:20.565549 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:20.565581 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:20.568268 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:20.568919 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:20.568925 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:20.568929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:20.569061 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:20.569456 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:20.569461 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:20.569465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:20.569735 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:20.569865 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:20.570514 (+ 649us) server_negotiation.cc:299] Negotiation successful
0810 00:51:20.570727 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":228,"threads_started":1}
WARNING: no policy specified for kudu/127.10.97.67@KRBTEST.COM; defaulting to no policy
I20260810 00:51:20.571985 11200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:20.554552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:41205 (local address 127.10.97.66:49825)
0810 00:51:20.555303 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:20.555355 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:20.557080 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:20.557755 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:20.557764 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:20.558298 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:20.558917 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:20.558936 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:20.560056 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:20.560060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:20.561252 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:20.561265 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:20.562077 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:20.562086 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:20.562296 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:20.562897 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:20.562922 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:20.565310 (+ 2388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:20.568398 (+ 3088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:20.568407 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:20.568421 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:20.568805 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:20.569166 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:20.569171 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:20.569175 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:20.569350 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:20.569859 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:20.569866 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:20.570181 (+ 315us) client_negotiation.cc:769] Sending connection context
0810 00:51:20.570405 (+ 224us) client_negotiation.cc:240] Negotiation successful
0810 00:51:20.570867 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":317,"threads_started":1}
I20260810 00:51:20.573305 11197 heartbeater.cc:347] Connected to a master server at 127.10.97.126:41205
I20260810 00:51:20.573565 11197 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:20.574222 11197 heartbeater.cc:511] Master 127.10.97.126:41205 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.67@KRBTEST.COM" created.
I20260810 00:51:20.575618 10706 ts_manager.cc:194] Registered new tserver with Master: e5e9aebf12624083a5d36d406554f237 (127.10.97.66:34375)
I20260810 00:51:20.576473 10706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:49825
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
I20260810 00:51:20.592847 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:41205
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:37439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:20.905627 11203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:20.905872 11203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:20.906003 11203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:20.912845 11203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:20.912943 11203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:20.913096 11203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:51:20.922241 11203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:37439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:41205
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:20.923322 11203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:20.924597 11203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:20.936506 11208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:20.936537 11213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:20.937000 11216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:20.937299 11203 server_base.cc:1034] running on GCE node
I20260810 00:51:20.937849 11203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:20.938700 11203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:20.939882 11203 hybrid_clock.cc:648] HybridClock initialized: now 1786323080939829 us; error 45 us; skew 500 ppm
I20260810 00:51:20.940166 11203 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:20 dist-test-slave-jlzn krb5kdc[10638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323080, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:20.943086 11203 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:51:20.944439 11203 webserver.cc:460] Webserver started at http://127.10.97.67:41085/ using document root <none> and password file <none>
I20260810 00:51:20.945050 11203 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:20.945149 11203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:20.945398 11203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:20.948166 11203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "3319c87abaf84d738157ce529c71a649"
format_stamp: "Formatted at 2026-08-10 00:51:20 on dist-test-slave-jlzn"
server_key: "17562e49cc65def20d6d5c330093329d"
server_key_iv: "5186fe2f6395dda064d7ad245f2b0355"
server_key_version: "encryptionkey@0"
I20260810 00:51:20.948866 11203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "3319c87abaf84d738157ce529c71a649"
format_stamp: "Formatted at 2026-08-10 00:51:20 on dist-test-slave-jlzn"
server_key: "17562e49cc65def20d6d5c330093329d"
server_key_iv: "5186fe2f6395dda064d7ad245f2b0355"
server_key_version: "encryptionkey@0"
I20260810 00:51:20.953614 11203 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:51:20.956784 11225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:20.957965 11203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:51:20.958101 11203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "3319c87abaf84d738157ce529c71a649"
format_stamp: "Formatted at 2026-08-10 00:51:20 on dist-test-slave-jlzn"
server_key: "17562e49cc65def20d6d5c330093329d"
server_key_iv: "5186fe2f6395dda064d7ad245f2b0355"
server_key_version: "encryptionkey@0"
I20260810 00:51:20.958276 11203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:20.984303 11203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:20.988185 11203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:20.988409 11203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:20.989225 11203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:20.990366 11203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:20.990437 11203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:20.990521 11203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:20.990568 11203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:21.022472 11203 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:39243
I20260810 00:51:21.022532 11408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:39243 every 8 connection(s)
I20260810 00:51:21.023587 11203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:51:21.025926 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 11203
I20260810 00:51:21.026139 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:51:21.026664 10629 external_mini_cluster.cc:1428] Setting key 3d7c0463e64ff4d8274776192ab918b7
Aug 10 00:51:21 dist-test-slave-jlzn krb5kdc[10638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323080, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:21.043313 11199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:21.026088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:33279 (local address 127.10.97.126:41205)
0810 00:51:21.026275 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:21.026279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:21.027722 (+ 1443us) server_negotiation.cc:407] Connection header received
0810 00:51:21.030273 (+ 2551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:21.030278 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:21.030375 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:21.030477 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:21.032413 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:21.033132 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:21.034637 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:21.035031 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:21.035119 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:21.038087 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:21.038092 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:21.038095 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:21.038126 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:21.040604 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:21.041208 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:21.041212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:21.041214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:21.041352 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:21.041756 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:21.041760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:21.041762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:21.042021 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:21.042178 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:21.042766 (+ 588us) server_negotiation.cc:299] Negotiation successful
0810 00:51:21.042984 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 00:51:21.044097 11411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:21.026647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:41205 (local address 127.10.97.67:33279)
0810 00:51:21.027542 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:21.027596 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:21.029980 (+ 2384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:21.030738 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:21.030750 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:21.031428 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:21.032246 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:21.032270 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:21.033284 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:21.033289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:21.034508 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:21.034525 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:21.035188 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:21.035193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:21.035373 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:21.035919 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:21.035940 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:21.037907 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:21.040697 (+ 2790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:21.040702 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:21.040712 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:21.041078 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:21.041457 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:21.041460 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:21.041463 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:21.041641 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:21.042144 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:21.042151 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:21.042430 (+ 279us) client_negotiation.cc:769] Sending connection context
0810 00:51:21.042649 (+ 219us) client_negotiation.cc:240] Negotiation successful
0810 00:51:21.043081 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":299,"threads_started":1}
I20260810 00:51:21.045271 11409 heartbeater.cc:347] Connected to a master server at 127.10.97.126:41205
I20260810 00:51:21.045499 11409 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:21.046087 11409 heartbeater.cc:511] Master 127.10.97.126:41205 requested a full tablet report, sending...
I20260810 00:51:21.047286 10706 ts_manager.cc:194] Registered new tserver with Master: 3319c87abaf84d738157ce529c71a649 (127.10.97.67:39243)
I20260810 00:51:21.048069 10706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:33279
I20260810 00:51:21.054450 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:51:21.062738 10994 heartbeater.cc:503] Master 127.10.97.126:41205 was elected leader, sending a full tablet report...
Aug 10 00:51:21 dist-test-slave-jlzn krb5kdc[10638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323081, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 00:51:21 dist-test-slave-jlzn krb5kdc[10638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323081, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:51:21.098621 11417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:21.085563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42406 (local address 127.10.97.65:33341)
0810 00:51:21.086124 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:21.086133 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:21.086157 (+ 24us) server_negotiation.cc:407] Connection header received
0810 00:51:21.086250 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:21.086256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:21.086441 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:21.086554 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:21.087779 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:21.088758 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:21.089914 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:21.090428 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:21.090515 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:21.093994 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:21.094004 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:21.094016 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:21.094051 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:21.096485 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:21.097010 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:21.097015 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:21.097017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:21.097129 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:21.097396 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:21.097399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:21.097401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:21.097713 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:21.097940 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:21.098101 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 00:51:21.098282 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":300,"threads_started":1}
W20260810 00:51:21.099929 10872 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42406
Aug 10 00:51:21 dist-test-slave-jlzn krb5kdc[10638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 00:51:21 dist-test-slave-jlzn krb5kdc[10638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:51:21.139425 11417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:21.127258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42410 (local address 127.10.97.65:33341)
0810 00:51:21.127450 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:21.127456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:21.127654 (+ 198us) server_negotiation.cc:407] Connection header received
0810 00:51:21.127773 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:21.127776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:21.127891 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:21.127970 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:21.129105 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:21.129761 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:21.130909 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:21.131333 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:21.131441 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:21.134898 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:21.134903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:21.134906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:21.134937 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:21.137202 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:21.137764 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:21.137768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:21.137770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:21.137885 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:21.138210 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:21.138215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:21.138218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:21.138442 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:21.138655 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:21.138943 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 00:51:21.139153 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 00:51:21.141109 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 10811
I20260810 00:51:21.151901 11417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:21.142029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42414 (local address 127.10.97.65:33341)
0810 00:51:21.142209 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:21.142214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:21.142362 (+ 148us) server_negotiation.cc:407] Connection header received
0810 00:51:21.142473 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:21.142477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:21.142527 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:21.142593 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:21.143595 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:21.144377 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:21.145506 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:21.145889 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:21.145970 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:21.147904 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:21.147909 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:21.147912 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:21.147936 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:21.150005 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:21.150530 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:21.150533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:21.150535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:21.150655 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:21.150899 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:21.150902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:21.150903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:21.151114 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:21.151271 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:21.151428 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 00:51:21.151569 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 00:51:21.152590 10986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:21.240658 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 10811
I20260810 00:51:21.266803 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 11005
Aug 10 00:51:21 dist-test-slave-jlzn krb5kdc[10638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.66@KRBTEST.COM
I20260810 00:51:21.284027 11429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:21.267694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43950 (local address 127.10.97.66:34375)
0810 00:51:21.268472 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:21.268480 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:21.268510 (+ 30us) server_negotiation.cc:407] Connection header received
0810 00:51:21.268580 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:21.268586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:21.268784 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:21.268937 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:21.270165 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:21.271216 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:21.272622 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:21.273505 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:21.273656 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:21.277580 (+ 3924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:21.277596 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:21.277614 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:21.277664 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:21.281231 (+ 3567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:21.281813 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:21.281822 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:21.281826 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:21.281993 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:21.282316 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:21.282320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:21.282324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:21.282760 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:21.283060 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:21.283281 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 00:51:21.283540 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":522,"threads_started":1}
I20260810 00:51:21.285328 11188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:21.365895 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 11005
I20260810 00:51:21.389835 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 11203
Aug 10 00:51:21 dist-test-slave-jlzn krb5kdc[10638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.67@KRBTEST.COM
I20260810 00:51:21.404656 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:21.390682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37318 (local address 127.10.97.67:39243)
0810 00:51:21.391289 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:21.391296 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:21.391326 (+ 30us) server_negotiation.cc:407] Connection header received
0810 00:51:21.391387 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:21.391394 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:21.391624 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:21.391752 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:21.392932 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:21.393929 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:21.395233 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:21.395777 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:21.395988 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:21.399714 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:21.399726 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:21.399740 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:21.399777 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:21.402297 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:21.402877 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:21.402883 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:21.402886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:21.403010 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:21.403270 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:21.403272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:21.403274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:21.403592 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:21.403853 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:21.404158 (+ 305us) server_negotiation.cc:299] Negotiation successful
0810 00:51:21.404370 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":282,"threads_started":1}
I20260810 00:51:21.405872 11401 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:21.492425 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 11203
I20260810 00:51:21.511322 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 10660
I20260810 00:51:21.512413 10764 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:21.610299 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 10660
2026-08-10T00:51:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4207 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:51:21 dist-test-slave-jlzn krb5kdc[11439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:51:21 dist-test-slave-jlzn krb5kdc[11439](info): set up 2 sockets
Aug 10 00:51:21 dist-test-slave-jlzn krb5kdc[11439](info): commencing operation
krb5kdc: starting...
W20260810 00:51:22.774863 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:51:22 dist-test-slave-jlzn krb5kdc[11439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:51:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:51:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:51:22.940598 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:32977
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:45141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:32977
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:23.239307 11455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:23.239550 11455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:23.239675 11455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:23.246443 11455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:51:23.246531 11455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:23.246603 11455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:23.246677 11455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:51:23.246737 11455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:51:23.255885 11455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:45141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:32977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:32977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:23.256939 11455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:23.258123 11455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:23.268594 11466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:23.268666 11464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:23.268651 11468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:23.269683 11455 server_base.cc:1034] running on GCE node
I20260810 00:51:23.270351 11455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:23.271598 11455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:23.272799 11455 hybrid_clock.cc:648] HybridClock initialized: now 1786323083272740 us; error 45 us; skew 500 ppm
I20260810 00:51:23.273093 11455 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:23 dist-test-slave-jlzn krb5kdc[11439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323083, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:23.275916 11455 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:51:23.277179 11455 webserver.cc:460] Webserver started at http://127.10.97.126:36319/ using document root <none> and password file <none>
I20260810 00:51:23.277801 11455 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:23.277913 11455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:23.278196 11455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:23.280938 11455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "e084ea96cd6c421681453e1bbb1efab7"
format_stamp: "Formatted at 2026-08-10 00:51:23 on dist-test-slave-jlzn"
server_key: "e2e5f5e5c8c4e006b99474b9a15bc8eb"
server_key_iv: "e8936eaf84d8eaf77135e09f142d651a"
server_key_version: "encryptionkey@0"
I20260810 00:51:23.281667 11455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "e084ea96cd6c421681453e1bbb1efab7"
format_stamp: "Formatted at 2026-08-10 00:51:23 on dist-test-slave-jlzn"
server_key: "e2e5f5e5c8c4e006b99474b9a15bc8eb"
server_key_iv: "e8936eaf84d8eaf77135e09f142d651a"
server_key_version: "encryptionkey@0"
I20260810 00:51:23.286239 11455 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260810 00:51:23.289346 11479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:23.290426 11455 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:51:23.290575 11455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "e084ea96cd6c421681453e1bbb1efab7"
format_stamp: "Formatted at 2026-08-10 00:51:23 on dist-test-slave-jlzn"
server_key: "e2e5f5e5c8c4e006b99474b9a15bc8eb"
server_key_iv: "e8936eaf84d8eaf77135e09f142d651a"
server_key_version: "encryptionkey@0"
I20260810 00:51:23.290743 11455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:23.307265 11455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:23.310580 11455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:23.310815 11455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:23.327231 11455 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:32977
I20260810 00:51:23.327268 11558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:32977 every 8 connection(s)
I20260810 00:51:23.328401 11455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:51:23.331774 11559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:23.332463 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 11455
I20260810 00:51:23.332671 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:51:23.333310 10629 external_mini_cluster.cc:1428] Setting key c8cfdfcfe2eeca2c93be5e938b71e2c1
I20260810 00:51:23.344667 11559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7: Bootstrap starting.
Aug 10 00:51:23 dist-test-slave-jlzn krb5kdc[11439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:23.347950 11559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:23.349191 11559 log.cc:824] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:23.352177 11559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7: No bootstrap required, opened a new log
I20260810 00:51:23.355710 11563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:23.335050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41168 (local address 127.10.97.126:32977)
0810 00:51:23.335782 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:23.335793 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:23.335861 (+ 68us) server_negotiation.cc:407] Connection header received
0810 00:51:23.337450 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:23.337474 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:23.337822 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:23.338123 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:23.339864 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:23.341155 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:23.342871 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:23.343471 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:23.343584 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:23.347916 (+ 4332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:23.347927 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:23.347950 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:23.347998 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:23.351537 (+ 3539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:23.352202 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:23.352210 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:23.352217 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:23.352417 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:23.352761 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:23.352766 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:23.352768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:23.353426 (+ 658us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:23.353722 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:23.354040 (+ 318us) server_negotiation.cc:299] Negotiation successful
0810 00:51:23.354438 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":307,"threads_started":1}
I20260810 00:51:23.361078 11559 raft_consensus.cc:348] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e084ea96cd6c421681453e1bbb1efab7" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 32977 } }
I20260810 00:51:23.361464 11559 raft_consensus.cc:374] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:23.361585 11559 raft_consensus.cc:729] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e084ea96cd6c421681453e1bbb1efab7, State: Initialized, Role: FOLLOWER
I20260810 00:51:23.362119 11559 consensus_queue.cc:260] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e084ea96cd6c421681453e1bbb1efab7" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 32977 } }
I20260810 00:51:23.362322 11559 raft_consensus.cc:388] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:51:23.362444 11559 raft_consensus.cc:482] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:51:23.362587 11559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:23.364578 11559 raft_consensus.cc:504] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e084ea96cd6c421681453e1bbb1efab7" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 32977 } }
I20260810 00:51:23.365032 11559 leader_election.cc:302] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e084ea96cd6c421681453e1bbb1efab7; no voters:
I20260810 00:51:23.365522 11559 leader_election.cc:288] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:51:23.365633 11565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:23.366744 11565 raft_consensus.cc:686] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7 [term 1 LEADER]: Becoming Leader. State: Replica: e084ea96cd6c421681453e1bbb1efab7, State: Running, Role: LEADER
I20260810 00:51:23.367192 11565 consensus_queue.cc:237] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e084ea96cd6c421681453e1bbb1efab7" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 32977 } }
I20260810 00:51:23.367372 11559 sys_catalog.cc:564] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:51:23.369757 11566 sys_catalog.cc:455] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e084ea96cd6c421681453e1bbb1efab7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e084ea96cd6c421681453e1bbb1efab7" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 32977 } } }
I20260810 00:51:23.369796 11567 sys_catalog.cc:455] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e084ea96cd6c421681453e1bbb1efab7. Latest consensus state: current_term: 1 leader_uuid: "e084ea96cd6c421681453e1bbb1efab7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e084ea96cd6c421681453e1bbb1efab7" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 32977 } } }
I20260810 00:51:23.370124 11567 sys_catalog.cc:458] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7 [sys.catalog]: This master's current role is: LEADER
I20260810 00:51:23.370134 11566 sys_catalog.cc:458] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7 [sys.catalog]: This master's current role is: LEADER
I20260810 00:51:23.371790 11575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:51:23.376194 11575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:51:23.384166 11575 catalog_manager.cc:1347] Generated new cluster ID: 9699e7f9d6634fe3a3e42f162a0fc392
I20260810 00:51:23.384256 11575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:51:23.438488 11575 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:51:23.440459 11575 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:51:23.460341 11575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e084ea96cd6c421681453e1bbb1efab7: Generated new TSK 0
I20260810 00:51:23.461056 11575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
I20260810 00:51:23.508611 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:32977
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:45141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:23.814157 11598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:23.814402 11598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:23.814532 11598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:23.821364 11598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:23.821463 11598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:23.821614 11598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:51:23.830621 11598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:45141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:32977
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:23.831766 11598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:23.833006 11598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:23.845172 11604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:23.845196 11607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:23.845122 11609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:23.845880 11598 server_base.cc:1034] running on GCE node
I20260810 00:51:23.846395 11598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:23.847190 11598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:23.848379 11598 hybrid_clock.cc:648] HybridClock initialized: now 1786323083848323 us; error 44 us; skew 500 ppm
I20260810 00:51:23.848673 11598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:23 dist-test-slave-jlzn krb5kdc[11439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323083, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:23.851730 11598 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:51:23.853025 11598 webserver.cc:460] Webserver started at http://127.10.97.65:34527/ using document root <none> and password file <none>
I20260810 00:51:23.853626 11598 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:23.853724 11598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:23.853968 11598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:23.856709 11598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "44fd1984bf104b39bec4364aa9a27cd2"
format_stamp: "Formatted at 2026-08-10 00:51:23 on dist-test-slave-jlzn"
server_key: "480a4cb694636b60d8cd92daf942c463"
server_key_iv: "b6fd042ded0d5a5ab72a950a6d290243"
server_key_version: "encryptionkey@0"
I20260810 00:51:23.857405 11598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "44fd1984bf104b39bec4364aa9a27cd2"
format_stamp: "Formatted at 2026-08-10 00:51:23 on dist-test-slave-jlzn"
server_key: "480a4cb694636b60d8cd92daf942c463"
server_key_iv: "b6fd042ded0d5a5ab72a950a6d290243"
server_key_version: "encryptionkey@0"
I20260810 00:51:23.862063 11598 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 00:51:23.865243 11620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:23.866394 11598 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:51:23.866546 11598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "44fd1984bf104b39bec4364aa9a27cd2"
format_stamp: "Formatted at 2026-08-10 00:51:23 on dist-test-slave-jlzn"
server_key: "480a4cb694636b60d8cd92daf942c463"
server_key_iv: "b6fd042ded0d5a5ab72a950a6d290243"
server_key_version: "encryptionkey@0"
I20260810 00:51:23.866719 11598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:23.891074 11598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:23.894901 11598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:23.895150 11598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:23.896005 11598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:23.897190 11598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:23.897262 11598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:23.897375 11598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:23.897425 11598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:23.926029 11598 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:38411
I20260810 00:51:23.926084 11803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:38411 every 8 connection(s)
I20260810 00:51:23.927116 11598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:51:23.931483 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 11598
I20260810 00:51:23.931699 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:51:23.932310 10629 external_mini_cluster.cc:1428] Setting key 6220669cbe49414af2e7b8f0d368ee49
Aug 10 00:51:23 dist-test-slave-jlzn krb5kdc[11439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323083, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:23.947888 11807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:23.929767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:32977 (local address 127.10.97.65:59327)
0810 00:51:23.930400 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:23.930434 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:23.932220 (+ 1786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:23.932897 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:23.932907 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:23.933496 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:23.934214 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:23.934241 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:23.935250 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:23.935255 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:23.936476 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:23.936493 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:23.937216 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:23.937225 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:23.937451 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:23.938026 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:23.938048 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:23.940373 (+ 2325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:23.943663 (+ 3290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:23.943672 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:23.943685 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:23.944308 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:23.944687 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:23.944692 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:23.944697 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:23.944935 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:23.945465 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:23.945473 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:23.945835 (+ 362us) client_negotiation.cc:769] Sending connection context
0810 00:51:23.946114 (+ 279us) client_negotiation.cc:240] Negotiation successful
0810 00:51:23.946621 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":245,"threads_started":1}
I20260810 00:51:23.948588 11806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:23.929322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:59327 (local address 127.10.97.126:32977)
0810 00:51:23.929724 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:23.929729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:23.930535 (+ 806us) server_negotiation.cc:407] Connection header received
0810 00:51:23.932465 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:23.932471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:23.932546 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:23.932642 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:23.934373 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:23.935096 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:23.936613 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:23.937015 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:23.937121 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:23.940592 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:23.940599 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:23.940603 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:23.940641 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:23.943473 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:23.944432 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:23.944438 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:23.944442 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:23.944568 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:23.945054 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:23.945060 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:23.945064 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:23.945339 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:23.945471 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:23.948032 (+ 2561us) server_negotiation.cc:299] Negotiation successful
0810 00:51:23.948246 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":204,"threads_started":1}
I20260810 00:51:23.951434 11804 heartbeater.cc:347] Connected to a master server at 127.10.97.126:32977
I20260810 00:51:23.951762 11804 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:23.952596 11804 heartbeater.cc:511] Master 127.10.97.126:32977 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.97.66@KRBTEST.COM; defaulting to no policy
I20260810 00:51:23.954739 11505 ts_manager.cc:194] Registered new tserver with Master: 44fd1984bf104b39bec4364aa9a27cd2 (127.10.97.65:38411)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.66@KRBTEST.COM" created.
I20260810 00:51:23.956578 11505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:59327
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
I20260810 00:51:23.976313 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:32977
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:45141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:24.277784 11811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:24.278020 11811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:24.278119 11811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:24.284907 11811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:24.284998 11811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:24.285111 11811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:51:24.294193 11811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:45141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:32977
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:24.295244 11811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:24.296388 11811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:24.308230 11819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:24.308319 11826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:24.308275 11818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:24.309186 11811 server_base.cc:1034] running on GCE node
I20260810 00:51:24.309758 11811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:24.310572 11811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:24.311770 11811 hybrid_clock.cc:648] HybridClock initialized: now 1786323084311723 us; error 42 us; skew 500 ppm
I20260810 00:51:24.312089 11811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:24 dist-test-slave-jlzn krb5kdc[11439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323084, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:24.315012 11811 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:51:24.316354 11811 webserver.cc:460] Webserver started at http://127.10.97.66:39973/ using document root <none> and password file <none>
I20260810 00:51:24.316929 11811 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:24.317039 11811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:24.317324 11811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:24.320048 11811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "d2b76bca035b4e58a4f64a087f119cb2"
format_stamp: "Formatted at 2026-08-10 00:51:24 on dist-test-slave-jlzn"
server_key: "cf33eea867f46054aa94199a4a5f1ccf"
server_key_iv: "f0b7954cda2748e3107f73ff8d8b0e64"
server_key_version: "encryptionkey@0"
I20260810 00:51:24.320773 11811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "d2b76bca035b4e58a4f64a087f119cb2"
format_stamp: "Formatted at 2026-08-10 00:51:24 on dist-test-slave-jlzn"
server_key: "cf33eea867f46054aa94199a4a5f1ccf"
server_key_iv: "f0b7954cda2748e3107f73ff8d8b0e64"
server_key_version: "encryptionkey@0"
I20260810 00:51:24.325466 11811 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:51:24.328675 11839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:24.329911 11811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:51:24.330056 11811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "d2b76bca035b4e58a4f64a087f119cb2"
format_stamp: "Formatted at 2026-08-10 00:51:24 on dist-test-slave-jlzn"
server_key: "cf33eea867f46054aa94199a4a5f1ccf"
server_key_iv: "f0b7954cda2748e3107f73ff8d8b0e64"
server_key_version: "encryptionkey@0"
I20260810 00:51:24.330236 11811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:24.345450 11811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:24.348820 11811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:24.349066 11811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:24.349931 11811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:24.351100 11811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:24.351186 11811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:24.351300 11811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:24.351358 11811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:24.380721 11811 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:36847
I20260810 00:51:24.380781 12026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:36847 every 8 connection(s)
I20260810 00:51:24.381790 11811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:51:24.390237 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 11811
I20260810 00:51:24.390410 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:51:24.390827 10629 external_mini_cluster.cc:1428] Setting key e519c4824dde4a7e80be33b0607536e5
Aug 10 00:51:24 dist-test-slave-jlzn krb5kdc[11439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323084, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:24.400851 11806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:24.384037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:52229 (local address 127.10.97.126:32977)
0810 00:51:24.384185 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:24.384190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:24.385249 (+ 1059us) server_negotiation.cc:407] Connection header received
0810 00:51:24.386981 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:24.386988 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:24.387080 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:24.387166 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:24.388622 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.389333 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.390873 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.391373 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.391514 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:24.395081 (+ 3567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:24.395086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:24.395091 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:24.395154 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:24.398094 (+ 2940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:24.398762 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:24.398767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:24.398769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:24.398930 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:24.399302 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:24.399307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:24.399310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:24.399594 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:24.399716 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:24.400469 (+ 753us) server_negotiation.cc:299] Negotiation successful
0810 00:51:24.400624 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 00:51:24.402058 12029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:24.384427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:32977 (local address 127.10.97.66:52229)
0810 00:51:24.385097 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:24.385136 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:24.386729 (+ 1593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:24.387393 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:24.387401 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:24.387937 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:24.388449 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:24.388466 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.389479 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.389483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:24.390737 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:24.390750 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.391598 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.391607 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:24.391907 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:24.392476 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:24.392497 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:24.394881 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:24.398224 (+ 3343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:24.398230 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:24.398240 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:24.398643 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:24.399022 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:24.399025 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:24.399027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:24.399200 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:24.399725 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:24.399733 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:24.400127 (+ 394us) client_negotiation.cc:769] Sending connection context
0810 00:51:24.400396 (+ 269us) client_negotiation.cc:240] Negotiation successful
0810 00:51:24.400877 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":265,"threads_started":1}
I20260810 00:51:24.403314 12027 heartbeater.cc:347] Connected to a master server at 127.10.97.126:32977
I20260810 00:51:24.403623 12027 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:24.404404 12027 heartbeater.cc:511] Master 127.10.97.126:32977 requested a full tablet report, sending...
I20260810 00:51:24.405884 11505 ts_manager.cc:194] Registered new tserver with Master: d2b76bca035b4e58a4f64a087f119cb2 (127.10.97.66:36847)
I20260810 00:51:24.407001 11505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:52229
WARNING: no policy specified for kudu/127.10.97.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
I20260810 00:51:24.432117 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:32977
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:45141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:24.736825 12034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:24.737066 12034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:24.737190 12034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:24.744024 12034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:24.744117 12034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:24.744264 12034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:51:24.753240 12034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:45141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:32977
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:24.754320 12034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:24.755528 12034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:24.767851 12042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:24.767823 12048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:24.767765 12044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:24.768545 12034 server_base.cc:1034] running on GCE node
I20260810 00:51:24.769119 12034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:24.769975 12034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:24.771173 12034 hybrid_clock.cc:648] HybridClock initialized: now 1786323084771134 us; error 32 us; skew 500 ppm
I20260810 00:51:24.771473 12034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:24 dist-test-slave-jlzn krb5kdc[11439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323084, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:24.774463 12034 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:51:24.775719 12034 webserver.cc:460] Webserver started at http://127.10.97.67:37125/ using document root <none> and password file <none>
I20260810 00:51:24.776347 12034 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:24.776448 12034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:24.776698 12034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:24.779310 12034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "fecd5a6b914240c7b1018f7a41b02c8b"
format_stamp: "Formatted at 2026-08-10 00:51:24 on dist-test-slave-jlzn"
server_key: "cfae4c793a4cfaa63ed653d1d5d3e177"
server_key_iv: "ca6b8bd3f3340c756c7ed15a975c9f7d"
server_key_version: "encryptionkey@0"
I20260810 00:51:24.780081 12034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "fecd5a6b914240c7b1018f7a41b02c8b"
format_stamp: "Formatted at 2026-08-10 00:51:24 on dist-test-slave-jlzn"
server_key: "cfae4c793a4cfaa63ed653d1d5d3e177"
server_key_iv: "ca6b8bd3f3340c756c7ed15a975c9f7d"
server_key_version: "encryptionkey@0"
I20260810 00:51:24.784705 12034 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:51:24.787761 12060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:24.788960 12034 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:51:24.789090 12034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "fecd5a6b914240c7b1018f7a41b02c8b"
format_stamp: "Formatted at 2026-08-10 00:51:24 on dist-test-slave-jlzn"
server_key: "cfae4c793a4cfaa63ed653d1d5d3e177"
server_key_iv: "ca6b8bd3f3340c756c7ed15a975c9f7d"
server_key_version: "encryptionkey@0"
I20260810 00:51:24.789242 12034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:24.840474 12034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:24.844198 12034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:24.844424 12034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:24.845333 12034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:24.846555 12034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:24.846632 12034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:24.846729 12034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:24.846772 12034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:24.880826 12034 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:38879
I20260810 00:51:24.880887 12236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:38879 every 8 connection(s)
I20260810 00:51:24.882066 12034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:51:24.885684 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 12034
I20260810 00:51:24.885890 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:51:24.886457 10629 external_mini_cluster.cc:1428] Setting key e58466531066d08c14fc79fbfff9cb5d
Aug 10 00:51:24 dist-test-slave-jlzn krb5kdc[11439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323084, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:24.900614 11806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:24.884336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:41217 (local address 127.10.97.126:32977)
0810 00:51:24.884540 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:24.884547 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:24.885744 (+ 1197us) server_negotiation.cc:407] Connection header received
0810 00:51:24.888033 (+ 2289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:24.888037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:24.888107 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:24.888194 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:24.889795 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.890517 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.892017 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.892502 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.892595 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:24.895754 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:24.895759 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:24.895762 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:24.895794 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:24.897947 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:24.898553 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:24.898556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:24.898559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:24.898668 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:24.898976 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:24.898979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:24.898981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:24.899233 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:24.899346 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:24.900041 (+ 695us) server_negotiation.cc:299] Negotiation successful
0810 00:51:24.900296 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 00:51:24.901425 12240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:24.884830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:32977 (local address 127.10.97.67:41217)
0810 00:51:24.885559 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:24.885606 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:24.887736 (+ 2130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:24.888442 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:24.888454 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:24.888966 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:24.889643 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:24.889662 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.890621 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.890631 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:24.891869 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:24.891884 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.892684 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.892689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:24.892880 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:24.893510 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:24.893531 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:24.895558 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:24.898067 (+ 2509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:24.898073 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:24.898083 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:24.898465 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:24.898738 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:24.898741 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:24.898743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:24.898904 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:24.899317 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:24.899322 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:24.899614 (+ 292us) client_negotiation.cc:769] Sending connection context
0810 00:51:24.899938 (+ 324us) client_negotiation.cc:240] Negotiation successful
0810 00:51:24.900391 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":251,"threads_started":1}
I20260810 00:51:24.902565 12237 heartbeater.cc:347] Connected to a master server at 127.10.97.126:32977
I20260810 00:51:24.902802 12237 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:24.903424 12237 heartbeater.cc:511] Master 127.10.97.126:32977 requested a full tablet report, sending...
I20260810 00:51:24.904690 11505 ts_manager.cc:194] Registered new tserver with Master: fecd5a6b914240c7b1018f7a41b02c8b (127.10.97.67:38879)
I20260810 00:51:24.905445 11505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:41217
I20260810 00:51:24.913075 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:51:24.931669 11806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:24.919617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41176 (local address 127.10.97.126:32977)
0810 00:51:24.919763 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:24.919767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:24.920269 (+ 502us) server_negotiation.cc:407] Connection header received
0810 00:51:24.920415 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:24.920420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:24.920476 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:24.920550 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:24.921954 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.922665 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.923923 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.924318 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.924446 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:24.927008 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:24.927012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:24.927016 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:24.927044 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:24.929428 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:24.930129 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:24.930132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:24.930134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:24.930239 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:24.930567 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:24.930570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:24.930572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:24.930832 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:24.931016 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:24.931242 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 00:51:24.931383 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 00:51:24.939081 11505 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41176:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:51:24.946834 11505 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:51:24.959769 11804 heartbeater.cc:503] Master 127.10.97.126:32977 was elected leader, sending a full tablet report...
I20260810 00:51:24.970908 12254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:24.961225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:38411 (local address 127.0.0.1:50800)
0810 00:51:24.962600 (+ 1375us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:24.962649 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:24.963058 (+ 409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:24.964062 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:24.964069 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:24.964117 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:24.964510 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:24.964524 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.966292 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.966296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:24.968122 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:24.968139 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.969944 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.969950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:24.970218 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:24.970274 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 00:51:24.970406 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 00:51:24.970594 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1125,"thread_start_us":975,"threads_started":1}
I20260810 00:51:24.971113 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:24.961467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50800 (local address 127.10.97.65:38411)
0810 00:51:24.962205 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:24.962214 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:24.962876 (+ 662us) server_negotiation.cc:407] Connection header received
0810 00:51:24.963327 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:24.963336 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:24.963601 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:24.963777 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:24.964708 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.966161 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.968323 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.969711 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.969836 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:24.970054 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:24.970450 (+ 396us) server_negotiation.cc:299] Negotiation successful
0810 00:51:24.970612 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":386,"threads_started":1}
I20260810 00:51:24.973186 12261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:24.963721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:38879 (local address 127.0.0.1:60286)
0810 00:51:24.964492 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:24.964508 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:24.964627 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:24.966016 (+ 1389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:24.966020 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:24.966049 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:24.966513 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:24.966542 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.968373 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.968379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:24.970493 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:24.970507 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.972358 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.972365 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:24.972550 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:24.972585 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 00:51:24.972672 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 00:51:24.972788 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":65,"thread_start_us":513,"threads_started":1}
I20260810 00:51:24.973225 12255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:24.961479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:36847 (local address 127.0.0.1:49546)
0810 00:51:24.963634 (+ 2155us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:24.963650 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:24.963749 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:24.965680 (+ 1931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:24.965686 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:24.965712 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:24.966095 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:24.966107 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.968021 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.968027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:24.969801 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:24.969816 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.972501 (+ 2685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.972507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:24.972650 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:24.972678 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 00:51:24.972747 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 00:51:24.972844 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2013,"mutex_wait_us":53,"thread_start_us":1869,"threads_started":1}
I20260810 00:51:24.973240 12263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:24.964307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60286 (local address 127.10.97.67:38879)
0810 00:51:24.965532 (+ 1225us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:24.965539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:24.965572 (+ 33us) server_negotiation.cc:407] Connection header received
0810 00:51:24.965652 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:24.965659 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:24.965845 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:24.966003 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:24.966739 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.968221 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.970675 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.972158 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.972334 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:24.972565 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:24.972710 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 00:51:24.972905 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1065,"thread_start_us":901,"threads_started":1}
I20260810 00:51:24.974128 12259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:24.962463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49546 (local address 127.10.97.66:36847)
0810 00:51:24.965098 (+ 2635us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:24.965107 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:24.965137 (+ 30us) server_negotiation.cc:407] Connection header received
0810 00:51:24.965229 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:24.965236 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:24.965448 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:24.965668 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:24.966311 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.967869 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.969986 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:24.972267 (+ 2281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:24.973586 (+ 1319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:24.973733 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:24.973783 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 00:51:24.973897 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2473,"thread_start_us":2375,"threads_started":1}
I20260810 00:51:24.975240 11698 tablet_service.cc:1467] Processing CreateTablet for tablet 29acc6a7fafc4fd0a8097ec107247ee1 (DEFAULT_TABLE table=test-table [id=f19342f7ddc04174914d312866fd7d07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:24.976944 11698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29acc6a7fafc4fd0a8097ec107247ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:24.977159 11919 tablet_service.cc:1467] Processing CreateTablet for tablet 29acc6a7fafc4fd0a8097ec107247ee1 (DEFAULT_TABLE table=test-table [id=f19342f7ddc04174914d312866fd7d07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:24.978147 11919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29acc6a7fafc4fd0a8097ec107247ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:24.977757 12132 tablet_service.cc:1467] Processing CreateTablet for tablet 29acc6a7fafc4fd0a8097ec107247ee1 (DEFAULT_TABLE table=test-table [id=f19342f7ddc04174914d312866fd7d07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:24.979099 12132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29acc6a7fafc4fd0a8097ec107247ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:24.988976 12266 tablet_bootstrap.cc:492] T 29acc6a7fafc4fd0a8097ec107247ee1 P d2b76bca035b4e58a4f64a087f119cb2: Bootstrap starting.
I20260810 00:51:24.990619 12267 tablet_bootstrap.cc:492] T 29acc6a7fafc4fd0a8097ec107247ee1 P 44fd1984bf104b39bec4364aa9a27cd2: Bootstrap starting.
I20260810 00:51:24.992309 12268 tablet_bootstrap.cc:492] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b: Bootstrap starting.
I20260810 00:51:24.994128 12267 tablet_bootstrap.cc:654] T 29acc6a7fafc4fd0a8097ec107247ee1 P 44fd1984bf104b39bec4364aa9a27cd2: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:24.994128 12266 tablet_bootstrap.cc:654] T 29acc6a7fafc4fd0a8097ec107247ee1 P d2b76bca035b4e58a4f64a087f119cb2: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:24.995316 12268 tablet_bootstrap.cc:654] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:24.995388 12267 log.cc:824] T 29acc6a7fafc4fd0a8097ec107247ee1 P 44fd1984bf104b39bec4364aa9a27cd2: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:24.995388 12266 log.cc:824] T 29acc6a7fafc4fd0a8097ec107247ee1 P d2b76bca035b4e58a4f64a087f119cb2: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:24.996447 12268 log.cc:824] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:24.998733 12266 tablet_bootstrap.cc:492] T 29acc6a7fafc4fd0a8097ec107247ee1 P d2b76bca035b4e58a4f64a087f119cb2: No bootstrap required, opened a new log
I20260810 00:51:24.998976 12266 ts_tablet_manager.cc:1397] T 29acc6a7fafc4fd0a8097ec107247ee1 P d2b76bca035b4e58a4f64a087f119cb2: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260810 00:51:24.999244 12268 tablet_bootstrap.cc:492] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b: No bootstrap required, opened a new log
I20260810 00:51:24.999500 12268 ts_tablet_manager.cc:1397] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20260810 00:51:25.000356 12267 tablet_bootstrap.cc:492] T 29acc6a7fafc4fd0a8097ec107247ee1 P 44fd1984bf104b39bec4364aa9a27cd2: No bootstrap required, opened a new log
I20260810 00:51:25.000623 12267 ts_tablet_manager.cc:1397] T 29acc6a7fafc4fd0a8097ec107247ee1 P 44fd1984bf104b39bec4364aa9a27cd2: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260810 00:51:25.006163 12266 raft_consensus.cc:348] T 29acc6a7fafc4fd0a8097ec107247ee1 P d2b76bca035b4e58a4f64a087f119cb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd1984bf104b39bec4364aa9a27cd2" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38411 } } peers { permanent_uuid: "d2b76bca035b4e58a4f64a087f119cb2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36847 } } peers { permanent_uuid: "fecd5a6b914240c7b1018f7a41b02c8b" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38879 } }
I20260810 00:51:25.006510 12266 raft_consensus.cc:374] T 29acc6a7fafc4fd0a8097ec107247ee1 P d2b76bca035b4e58a4f64a087f119cb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:25.006599 12266 raft_consensus.cc:729] T 29acc6a7fafc4fd0a8097ec107247ee1 P d2b76bca035b4e58a4f64a087f119cb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2b76bca035b4e58a4f64a087f119cb2, State: Initialized, Role: FOLLOWER
I20260810 00:51:25.007138 12266 consensus_queue.cc:260] T 29acc6a7fafc4fd0a8097ec107247ee1 P d2b76bca035b4e58a4f64a087f119cb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd1984bf104b39bec4364aa9a27cd2" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38411 } } peers { permanent_uuid: "d2b76bca035b4e58a4f64a087f119cb2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36847 } } peers { permanent_uuid: "fecd5a6b914240c7b1018f7a41b02c8b" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38879 } }
I20260810 00:51:25.008286 12027 heartbeater.cc:503] Master 127.10.97.126:32977 was elected leader, sending a full tablet report...
I20260810 00:51:25.008839 12266 ts_tablet_manager.cc:1428] T 29acc6a7fafc4fd0a8097ec107247ee1 P d2b76bca035b4e58a4f64a087f119cb2: Time spent starting tablet: real 0.010s user 0.008s sys 0.003s
I20260810 00:51:25.009419 12268 raft_consensus.cc:348] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd1984bf104b39bec4364aa9a27cd2" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38411 } } peers { permanent_uuid: "d2b76bca035b4e58a4f64a087f119cb2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36847 } } peers { permanent_uuid: "fecd5a6b914240c7b1018f7a41b02c8b" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38879 } }
I20260810 00:51:25.009794 12268 raft_consensus.cc:374] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:25.009889 12268 raft_consensus.cc:729] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecd5a6b914240c7b1018f7a41b02c8b, State: Initialized, Role: FOLLOWER
I20260810 00:51:25.010434 12268 consensus_queue.cc:260] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd1984bf104b39bec4364aa9a27cd2" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38411 } } peers { permanent_uuid: "d2b76bca035b4e58a4f64a087f119cb2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36847 } } peers { permanent_uuid: "fecd5a6b914240c7b1018f7a41b02c8b" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38879 } }
I20260810 00:51:25.010406 12267 raft_consensus.cc:348] T 29acc6a7fafc4fd0a8097ec107247ee1 P 44fd1984bf104b39bec4364aa9a27cd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd1984bf104b39bec4364aa9a27cd2" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38411 } } peers { permanent_uuid: "d2b76bca035b4e58a4f64a087f119cb2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36847 } } peers { permanent_uuid: "fecd5a6b914240c7b1018f7a41b02c8b" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38879 } }
I20260810 00:51:25.010789 12267 raft_consensus.cc:374] T 29acc6a7fafc4fd0a8097ec107247ee1 P 44fd1984bf104b39bec4364aa9a27cd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:25.010888 12267 raft_consensus.cc:729] T 29acc6a7fafc4fd0a8097ec107247ee1 P 44fd1984bf104b39bec4364aa9a27cd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44fd1984bf104b39bec4364aa9a27cd2, State: Initialized, Role: FOLLOWER
I20260810 00:51:25.011564 12237 heartbeater.cc:503] Master 127.10.97.126:32977 was elected leader, sending a full tablet report...
I20260810 00:51:25.012275 12268 ts_tablet_manager.cc:1428] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260810 00:51:25.012715 12267 consensus_queue.cc:260] T 29acc6a7fafc4fd0a8097ec107247ee1 P 44fd1984bf104b39bec4364aa9a27cd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd1984bf104b39bec4364aa9a27cd2" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38411 } } peers { permanent_uuid: "d2b76bca035b4e58a4f64a087f119cb2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36847 } } peers { permanent_uuid: "fecd5a6b914240c7b1018f7a41b02c8b" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38879 } }
I20260810 00:51:25.014539 12267 ts_tablet_manager.cc:1428] T 29acc6a7fafc4fd0a8097ec107247ee1 P 44fd1984bf104b39bec4364aa9a27cd2: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
I20260810 00:51:25.063694 12273 raft_consensus.cc:482] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:51:25.063931 12273 raft_consensus.cc:504] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd1984bf104b39bec4364aa9a27cd2" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38411 } } peers { permanent_uuid: "d2b76bca035b4e58a4f64a087f119cb2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36847 } } peers { permanent_uuid: "fecd5a6b914240c7b1018f7a41b02c8b" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38879 } }
I20260810 00:51:25.065760 12273 leader_election.cc:288] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44fd1984bf104b39bec4364aa9a27cd2 (127.10.97.65:38411), d2b76bca035b4e58a4f64a087f119cb2 (127.10.97.66:36847)
I20260810 00:51:25.072130 12240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:25.066037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:36847 (local address 127.10.97.67:40135)
0810 00:51:25.066361 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:25.066380 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:25.066490 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:25.066874 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:25.066883 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:25.066911 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:25.067104 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:25.067115 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.068363 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.068367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:25.070262 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:25.070278 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.071560 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.071565 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:25.071698 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:25.071723 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 00:51:25.071796 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 00:51:25.071938 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":39}
I20260810 00:51:25.072239 12259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:25.066284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:40135 (local address 127.10.97.66:36847)
0810 00:51:25.066475 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:25.066481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:25.066500 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:51:25.066679 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:25.066685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:25.066754 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:25.066836 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:25.067246 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.068239 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.070412 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.071396 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.071545 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:25.071616 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:25.071865 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 00:51:25.071991 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 00:51:25.073196 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:25.066208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:38411 (local address 127.10.97.67:37131)
0810 00:51:25.066893 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:25.066910 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:25.067029 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:25.067320 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:25.067323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:25.067342 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:25.067603 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:25.067610 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.068821 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.068827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:25.070741 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:25.070751 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.072506 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.072512 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:25.072684 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:25.072716 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 00:51:25.072811 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 00:51:25.072932 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"mutex_wait_us":22,"thread_start_us":389,"threads_started":1}
I20260810 00:51:25.073203 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:25.066148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:37131 (local address 127.10.97.65:38411)
0810 00:51:25.066332 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:25.066339 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:25.066989 (+ 650us) server_negotiation.cc:407] Connection header received
0810 00:51:25.067145 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:25.067148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:25.067212 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:25.067302 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:25.067741 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.068677 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.070870 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.072141 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.072292 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:25.072358 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:25.072850 (+ 492us) server_negotiation.cc:299] Negotiation successful
0810 00:51:25.072976 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 00:51:25.073455 11961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29acc6a7fafc4fd0a8097ec107247ee1" candidate_uuid: "fecd5a6b914240c7b1018f7a41b02c8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2b76bca035b4e58a4f64a087f119cb2" is_pre_election: true
I20260810 00:51:25.074085 11961 raft_consensus.cc:2454] T 29acc6a7fafc4fd0a8097ec107247ee1 P d2b76bca035b4e58a4f64a087f119cb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecd5a6b914240c7b1018f7a41b02c8b in term 0.
I20260810 00:51:25.074723 12065 leader_election.cc:302] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2b76bca035b4e58a4f64a087f119cb2, fecd5a6b914240c7b1018f7a41b02c8b; no voters:
I20260810 00:51:25.075100 12273 raft_consensus.cc:2781] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:51:25.075229 12273 raft_consensus.cc:482] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:51:25.075083 11732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29acc6a7fafc4fd0a8097ec107247ee1" candidate_uuid: "fecd5a6b914240c7b1018f7a41b02c8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44fd1984bf104b39bec4364aa9a27cd2" is_pre_election: true
I20260810 00:51:25.075321 12273 raft_consensus.cc:3037] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:25.075520 11732 raft_consensus.cc:2454] T 29acc6a7fafc4fd0a8097ec107247ee1 P 44fd1984bf104b39bec4364aa9a27cd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecd5a6b914240c7b1018f7a41b02c8b in term 0.
I20260810 00:51:25.077533 12273 raft_consensus.cc:504] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd1984bf104b39bec4364aa9a27cd2" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38411 } } peers { permanent_uuid: "d2b76bca035b4e58a4f64a087f119cb2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36847 } } peers { permanent_uuid: "fecd5a6b914240c7b1018f7a41b02c8b" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38879 } }
I20260810 00:51:25.078045 12273 leader_election.cc:288] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b [CANDIDATE]: Term 1 election: Requested vote from peers 44fd1984bf104b39bec4364aa9a27cd2 (127.10.97.65:38411), d2b76bca035b4e58a4f64a087f119cb2 (127.10.97.66:36847)
I20260810 00:51:25.078413 11732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29acc6a7fafc4fd0a8097ec107247ee1" candidate_uuid: "fecd5a6b914240c7b1018f7a41b02c8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44fd1984bf104b39bec4364aa9a27cd2"
I20260810 00:51:25.078485 11961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29acc6a7fafc4fd0a8097ec107247ee1" candidate_uuid: "fecd5a6b914240c7b1018f7a41b02c8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2b76bca035b4e58a4f64a087f119cb2"
I20260810 00:51:25.078567 11732 raft_consensus.cc:3037] T 29acc6a7fafc4fd0a8097ec107247ee1 P 44fd1984bf104b39bec4364aa9a27cd2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:25.078676 11961 raft_consensus.cc:3037] T 29acc6a7fafc4fd0a8097ec107247ee1 P d2b76bca035b4e58a4f64a087f119cb2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:25.080737 11732 raft_consensus.cc:2454] T 29acc6a7fafc4fd0a8097ec107247ee1 P 44fd1984bf104b39bec4364aa9a27cd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecd5a6b914240c7b1018f7a41b02c8b in term 1.
I20260810 00:51:25.080847 11961 raft_consensus.cc:2454] T 29acc6a7fafc4fd0a8097ec107247ee1 P d2b76bca035b4e58a4f64a087f119cb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecd5a6b914240c7b1018f7a41b02c8b in term 1.
I20260810 00:51:25.081081 12067 leader_election.cc:302] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44fd1984bf104b39bec4364aa9a27cd2, fecd5a6b914240c7b1018f7a41b02c8b; no voters:
I20260810 00:51:25.081399 12273 raft_consensus.cc:2781] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:25.081789 12273 raft_consensus.cc:686] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b [term 1 LEADER]: Becoming Leader. State: Replica: fecd5a6b914240c7b1018f7a41b02c8b, State: Running, Role: LEADER
I20260810 00:51:25.082243 12273 consensus_queue.cc:237] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd1984bf104b39bec4364aa9a27cd2" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38411 } } peers { permanent_uuid: "d2b76bca035b4e58a4f64a087f119cb2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36847 } } peers { permanent_uuid: "fecd5a6b914240c7b1018f7a41b02c8b" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38879 } }
I20260810 00:51:25.086905 11505 catalog_manager.cc:5495] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b reported cstate change: term changed from 0 to 1, leader changed from <none> to fecd5a6b914240c7b1018f7a41b02c8b (127.10.97.67). New cstate: current_term: 1 leader_uuid: "fecd5a6b914240c7b1018f7a41b02c8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd1984bf104b39bec4364aa9a27cd2" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2b76bca035b4e58a4f64a087f119cb2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecd5a6b914240c7b1018f7a41b02c8b" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38879 } health_report { overall_health: HEALTHY } } }
Aug 10 00:51:25 dist-test-slave-jlzn krb5kdc[11439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:51:25.129769 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:25.116364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50804 (local address 127.10.97.65:38411)
0810 00:51:25.116508 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:25.116513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:25.117018 (+ 505us) server_negotiation.cc:407] Connection header received
0810 00:51:25.117143 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:25.117147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:25.117204 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:25.117322 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:25.118694 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.119405 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.120531 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.120927 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.121014 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:25.124577 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:25.124589 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:25.124604 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:25.124640 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:25.127252 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:25.127882 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:25.127889 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:25.127891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:25.128025 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:25.128369 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:25.128372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:25.128374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:25.128699 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:25.128975 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:25.129261 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 00:51:25.129465 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260810 00:51:25.133471 12238 tablet.cc:2369] T 29acc6a7fafc4fd0a8097ec107247ee1 P fecd5a6b914240c7b1018f7a41b02c8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:51:25.133545 12028 tablet.cc:2369] T 29acc6a7fafc4fd0a8097ec107247ee1 P d2b76bca035b4e58a4f64a087f119cb2: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 00:51:25 dist-test-slave-jlzn krb5kdc[11439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323085, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 00:51:25 dist-test-slave-jlzn krb5kdc[11439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323085, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:51:25.176389 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:25.163374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50810 (local address 127.10.97.65:38411)
0810 00:51:25.163525 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:25.163530 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:25.163990 (+ 460us) server_negotiation.cc:407] Connection header received
0810 00:51:25.164161 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:25.164167 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:25.164233 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:25.164311 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:25.165564 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.166287 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.167594 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.168075 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.168191 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:25.171723 (+ 3532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:25.171728 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:25.171731 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:25.171762 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:25.174064 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:25.174623 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:25.174628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:25.174630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:25.174754 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:25.175113 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:25.175118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:25.175121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:25.175367 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:25.175597 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:25.175877 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 00:51:25.176085 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 00:51:25.179154 11805 tablet.cc:2369] T 29acc6a7fafc4fd0a8097ec107247ee1 P 44fd1984bf104b39bec4364aa9a27cd2: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 00:51:25 dist-test-slave-jlzn krb5kdc[11439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 00:51:25 dist-test-slave-jlzn krb5kdc[11439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:51:25.218353 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:25.205746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50826 (local address 127.10.97.65:38411)
0810 00:51:25.205917 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:25.205923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:25.206205 (+ 282us) server_negotiation.cc:407] Connection header received
0810 00:51:25.206358 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:25.206362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:25.206420 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:25.206499 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:25.207753 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.208484 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.209680 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.210096 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.210210 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:25.213636 (+ 3426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:25.213641 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:25.213645 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:25.213684 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:25.216040 (+ 2356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:25.216594 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:25.216598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:25.216600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:25.216722 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:25.217040 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:25.217045 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:25.217048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:25.217330 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:25.217552 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:25.217825 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 00:51:25.218041 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260810 00:51:25.220439 11663 tablet_service.cc:2908] Rejecting scan request for tablet 29acc6a7fafc4fd0a8097ec107247ee1: Uninitialized: clean time has not yet been initialized
I20260810 00:51:25.223654 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 11598
I20260810 00:51:25.234772 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:25.224583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50840 (local address 127.10.97.65:38411)
0810 00:51:25.224750 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:25.224756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:25.224974 (+ 218us) server_negotiation.cc:407] Connection header received
0810 00:51:25.225114 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:25.225119 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:25.225196 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:25.225270 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:25.226322 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.226942 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.228112 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.228519 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.228635 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:25.230467 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:25.230472 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:25.230475 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:25.230502 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:25.232534 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:25.233121 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:25.233126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:25.233130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:25.233251 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:25.233609 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:25.233613 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:25.233616 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:25.233930 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:25.234181 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:25.234340 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 00:51:25.234501 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 00:51:25.235353 11797 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:25.320868 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 11598
I20260810 00:51:25.350430 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 11811
Aug 10 00:51:25 dist-test-slave-jlzn krb5kdc[11439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.66@KRBTEST.COM
I20260810 00:51:25.364048 12259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:25.351148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49562 (local address 127.10.97.66:36847)
0810 00:51:25.351329 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:25.351335 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:25.351355 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:51:25.351482 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:25.351487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:25.351563 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:25.351655 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:25.352822 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.353563 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.354789 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.355195 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.355302 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:25.358976 (+ 3674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:25.358989 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:25.359004 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:25.359041 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:25.361716 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:25.362259 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:25.362266 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:25.362268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:25.362390 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:25.362662 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:25.362665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:25.362667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:25.363005 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:25.363249 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:25.363546 (+ 297us) server_negotiation.cc:299] Negotiation successful
0810 00:51:25.363712 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 00:51:25.364638 12019 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:25.448354 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 11811
I20260810 00:51:25.471370 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 12034
Aug 10 00:51:25 dist-test-slave-jlzn krb5kdc[11439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.67@KRBTEST.COM
I20260810 00:51:25.485229 12263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:25.472159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60290 (local address 127.10.97.67:38879)
0810 00:51:25.472339 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:25.472344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:25.472367 (+ 23us) server_negotiation.cc:407] Connection header received
0810 00:51:25.472562 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:25.472568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:25.472673 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:25.472757 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:25.473960 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.474700 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.475962 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:25.476371 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:25.476481 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:25.480305 (+ 3824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:25.480320 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:25.480337 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:25.480371 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:25.482923 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:25.483478 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:25.483485 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:25.483487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:25.483620 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:25.484043 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:25.484047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:25.484048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:25.484395 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:25.484613 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:25.484809 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 00:51:25.484981 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 00:51:25.485823 12229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:25.571024 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 12034
I20260810 00:51:25.593115 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 11455
I20260810 00:51:25.594170 11554 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:25.696628 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 11455
2026-08-10T00:51:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4085 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:51:25 dist-test-slave-jlzn krb5kdc[12308](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:51:25 dist-test-slave-jlzn krb5kdc[12308](info): set up 2 sockets
Aug 10 00:51:25 dist-test-slave-jlzn krb5kdc[12308](info): commencing operation
krb5kdc: starting...
W20260810 00:51:26.859004 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:51:26 dist-test-slave-jlzn krb5kdc[12308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:51:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:51:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:51:27.027202 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:43905
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:35019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:43905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:27.326413 12328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:27.326661 12328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:27.326753 12328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:27.333479 12328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:51:27.333570 12328 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 00:51:27.333616 12328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:27.333674 12328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:27.333721 12328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:51:27.333778 12328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:51:27.342763 12328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:35019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:43905
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:43905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:27.343730 12328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:27.344882 12328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:27.355373 12336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:27.355336 12338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:27.355412 12335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:27.356222 12328 server_base.cc:1034] running on GCE node
I20260810 00:51:27.356853 12328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:27.358024 12328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:27.359251 12328 hybrid_clock.cc:648] HybridClock initialized: now 1786323087359186 us; error 39 us; skew 500 ppm
I20260810 00:51:27.359539 12328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:27 dist-test-slave-jlzn krb5kdc[12308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323087, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:27.362371 12328 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:51:27.363600 12328 webserver.cc:460] Webserver started at http://127.10.97.126:33135/ using document root <none> and password file <none>
I20260810 00:51:27.364257 12328 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:27.364355 12328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:27.364642 12328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:27.367283 12328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "1a768f4ee7274d0ab66c4556cf0478dd"
format_stamp: "Formatted at 2026-08-10 00:51:27 on dist-test-slave-jlzn"
server_key: "5c6269a25b572e739cc38cd039ff3e9e"
server_key_iv: "1ba2b2c70501e15d0d7ff9757f4f15fd"
server_key_version: "encryptionkey@0"
I20260810 00:51:27.368014 12328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "1a768f4ee7274d0ab66c4556cf0478dd"
format_stamp: "Formatted at 2026-08-10 00:51:27 on dist-test-slave-jlzn"
server_key: "5c6269a25b572e739cc38cd039ff3e9e"
server_key_iv: "1ba2b2c70501e15d0d7ff9757f4f15fd"
server_key_version: "encryptionkey@0"
I20260810 00:51:27.372694 12328 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 00:51:27.375698 12346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:27.376832 12328 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:51:27.376973 12328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "1a768f4ee7274d0ab66c4556cf0478dd"
format_stamp: "Formatted at 2026-08-10 00:51:27 on dist-test-slave-jlzn"
server_key: "5c6269a25b572e739cc38cd039ff3e9e"
server_key_iv: "1ba2b2c70501e15d0d7ff9757f4f15fd"
server_key_version: "encryptionkey@0"
I20260810 00:51:27.377158 12328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:27.389258 12328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:27.392632 12328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:27.392861 12328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:27.412009 12328 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:43905
I20260810 00:51:27.412026 12437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:43905 every 8 connection(s)
I20260810 00:51:27.413172 12328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:51:27.416473 12439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:27.419358 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 12328
I20260810 00:51:27.419584 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:51:27.420413 10629 external_mini_cluster.cc:1428] Setting key 76484388717d0459b6e9a6fa13d514b4
I20260810 00:51:27.428347 12439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd: Bootstrap starting.
I20260810 00:51:27.431751 12439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:27.432976 12439 log.cc:824] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd: Log is configured to *not* fsync() on all Append() calls
Aug 10 00:51:27 dist-test-slave-jlzn krb5kdc[12308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:27.435575 12439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd: No bootstrap required, opened a new log
I20260810 00:51:27.442032 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:27.422278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39246 (local address 127.10.97.126:43905)
0810 00:51:27.422957 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:27.422967 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:27.423005 (+ 38us) server_negotiation.cc:407] Connection header received
0810 00:51:27.424662 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:27.424686 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:27.425027 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:27.425349 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:27.426970 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:27.428211 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:27.429864 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:27.430483 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:27.430608 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:27.435161 (+ 4553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:27.435169 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:27.435182 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:27.435213 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:27.438024 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:27.438641 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:27.438648 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:27.438653 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:27.438816 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:27.439160 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:27.439163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:27.439166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:27.439673 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:27.439979 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:27.440401 (+ 422us) server_negotiation.cc:299] Negotiation successful
0810 00:51:27.440814 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":315,"threads_started":1}
I20260810 00:51:27.443590 12439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a768f4ee7274d0ab66c4556cf0478dd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43905 } }
I20260810 00:51:27.443929 12439 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:27.444024 12439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a768f4ee7274d0ab66c4556cf0478dd, State: Initialized, Role: FOLLOWER
I20260810 00:51:27.444576 12439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a768f4ee7274d0ab66c4556cf0478dd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43905 } }
I20260810 00:51:27.444787 12439 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:51:27.444916 12439 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:51:27.445050 12439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:27.446970 12439 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a768f4ee7274d0ab66c4556cf0478dd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43905 } }
I20260810 00:51:27.447415 12439 leader_election.cc:302] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a768f4ee7274d0ab66c4556cf0478dd; no voters:
I20260810 00:51:27.447989 12439 leader_election.cc:288] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:51:27.448086 12446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:27.448701 12446 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd [term 1 LEADER]: Becoming Leader. State: Replica: 1a768f4ee7274d0ab66c4556cf0478dd, State: Running, Role: LEADER
I20260810 00:51:27.449229 12446 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a768f4ee7274d0ab66c4556cf0478dd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43905 } }
I20260810 00:51:27.450052 12439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:51:27.452088 12448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a768f4ee7274d0ab66c4556cf0478dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a768f4ee7274d0ab66c4556cf0478dd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43905 } } }
I20260810 00:51:27.452488 12448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd [sys.catalog]: This master's current role is: LEADER
I20260810 00:51:27.453011 12449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a768f4ee7274d0ab66c4556cf0478dd. Latest consensus state: current_term: 1 leader_uuid: "1a768f4ee7274d0ab66c4556cf0478dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a768f4ee7274d0ab66c4556cf0478dd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43905 } } }
I20260810 00:51:27.453290 12449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd [sys.catalog]: This master's current role is: LEADER
I20260810 00:51:27.454690 12460 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:51:27.459450 12460 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:51:27.467176 12460 catalog_manager.cc:1347] Generated new cluster ID: 4ba73a2c53d549479f109ccedacca521
I20260810 00:51:27.467265 12460 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:51:27.474709 12460 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:51:27.476009 12460 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:51:27.483227 12460 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd: Generated new TSK 0
I20260810 00:51:27.483891 12460 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
I20260810 00:51:27.532531 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:43905
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:35019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:27.842577 12476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:27.842826 12476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:27.842959 12476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:27.849926 12476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:27.850033 12476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:27.850124 12476 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 00:51:27.850205 12476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:51:27.859447 12476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:35019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.97.126:43905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:27.860519 12476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:27.861725 12476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:27.873486 12484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:27.873584 12485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:27.873539 12487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:27.874557 12476 server_base.cc:1034] running on GCE node
I20260810 00:51:27.875162 12476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:27.876029 12476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:27.877238 12476 hybrid_clock.cc:648] HybridClock initialized: now 1786323087877170 us; error 47 us; skew 500 ppm
I20260810 00:51:27.877492 12476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:27 dist-test-slave-jlzn krb5kdc[12308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323087, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:27.880430 12476 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:51:27.881716 12476 webserver.cc:460] Webserver started at http://127.10.97.65:42273/ using document root <none> and password file <none>
I20260810 00:51:27.882320 12476 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:27.882418 12476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:27.882705 12476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:27.885469 12476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "e3c5dce1cb384204acc347d4b3e2a29d"
format_stamp: "Formatted at 2026-08-10 00:51:27 on dist-test-slave-jlzn"
server_key: "5e71bbb58dc02fb9074ef35a71359a6e"
server_key_iv: "d480dc988084cc8fbd7fcf47166cc1eb"
server_key_version: "encryptionkey@0"
I20260810 00:51:27.886205 12476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "e3c5dce1cb384204acc347d4b3e2a29d"
format_stamp: "Formatted at 2026-08-10 00:51:27 on dist-test-slave-jlzn"
server_key: "5e71bbb58dc02fb9074ef35a71359a6e"
server_key_iv: "d480dc988084cc8fbd7fcf47166cc1eb"
server_key_version: "encryptionkey@0"
I20260810 00:51:27.890926 12476 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 00:51:27.894073 12495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:27.895229 12476 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:51:27.895375 12476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "e3c5dce1cb384204acc347d4b3e2a29d"
format_stamp: "Formatted at 2026-08-10 00:51:27 on dist-test-slave-jlzn"
server_key: "5e71bbb58dc02fb9074ef35a71359a6e"
server_key_iv: "d480dc988084cc8fbd7fcf47166cc1eb"
server_key_version: "encryptionkey@0"
I20260810 00:51:27.895556 12476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:27.922797 12476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:27.926190 12476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:27.926424 12476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:27.929003 12476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:27.929102 12476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:27.929268 12476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:27.929349 12476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 00:51:27 dist-test-slave-jlzn krb5kdc[12308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323087, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:27.948997 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:27.929399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:58387 (local address 127.10.97.126:43905)
0810 00:51:27.929582 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:27.929589 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:27.930834 (+ 1245us) server_negotiation.cc:407] Connection header received
0810 00:51:27.932751 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:27.932756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:27.932836 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:27.932920 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:27.934944 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:27.935926 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:27.938295 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:27.938858 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:27.939000 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:27.943561 (+ 4561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:27.943567 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:27.943572 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:27.943604 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:27.946045 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:27.946719 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:27.946725 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:27.946729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:27.946845 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:27.947338 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:27.947342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:27.947343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:27.947607 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:27.947737 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:27.948457 (+ 720us) server_negotiation.cc:299] Negotiation successful
0810 00:51:27.948680 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 00:51:27.950125 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:27.929934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43905 (local address 127.10.97.65:58387)
0810 00:51:27.930691 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:27.930729 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:27.932483 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:27.933154 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:27.933161 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:27.933833 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:27.934691 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:27.934727 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:27.936087 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:27.936090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:27.938158 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:27.938177 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:27.939170 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:27.939180 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:27.939517 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:27.940277 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:27.940313 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:27.943348 (+ 3035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:27.946180 (+ 2832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:27.946188 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:27.946201 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:27.946611 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:27.946987 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:27.946992 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:27.946996 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:27.947231 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:27.947736 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:27.947743 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:27.948113 (+ 370us) client_negotiation.cc:769] Sending connection context
0810 00:51:27.948362 (+ 249us) client_negotiation.cc:240] Negotiation successful
0810 00:51:27.948852 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":304,"threads_started":1}
I20260810 00:51:27.970072 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:27.956897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:46959 (local address 127.10.97.126:43905)
0810 00:51:27.957052 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:27.957057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:27.957700 (+ 643us) server_negotiation.cc:407] Connection header received
0810 00:51:27.957844 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:27.957849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:27.957913 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:27.957986 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:27.958820 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:27.959500 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:27.961381 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:27.961859 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:27.961980 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:27.964347 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:27.964352 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:27.964355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:27.964382 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:27.967017 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:27.967990 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:27.967995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:27.967999 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:27.968153 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:27.968565 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:27.968570 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:27.968573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:27.968852 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:27.968987 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:27.969538 (+ 551us) server_negotiation.cc:299] Negotiation successful
0810 00:51:27.969744 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 00:51:27.970176 12618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:27.956866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43905 (local address 127.10.97.65:46959)
0810 00:51:27.957615 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:27.957633 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:27.957746 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:27.958020 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:27.958025 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:27.958428 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:27.958693 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:27.958701 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:27.959658 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:27.959663 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:27.961242 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:27.961255 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:27.962224 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:27.962231 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:27.962410 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:27.963007 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:27.963035 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:27.964190 (+ 1155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:27.967166 (+ 2976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:27.967171 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:27.967179 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:27.967790 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:27.968249 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:27.968257 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:27.968261 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:27.968414 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:27.969062 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:27.969071 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:27.969276 (+ 205us) client_negotiation.cc:769] Sending connection context
0810 00:51:27.969411 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 00:51:27.969726 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":447,"threads_started":1}
I20260810 00:51:27.974584 12476 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:39961
I20260810 00:51:27.974624 12679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:39961 every 8 connection(s)
I20260810 00:51:27.975664 12476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:51:27.977560 12680 heartbeater.cc:347] Connected to a master server at 127.10.97.126:43905
I20260810 00:51:27.977806 12680 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:27.978017 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 12476
I20260810 00:51:27.978248 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:51:27.978543 12680 heartbeater.cc:511] Master 127.10.97.126:43905 requested a full tablet report, sending...
I20260810 00:51:27.979065 10629 external_mini_cluster.cc:1428] Setting key 745b919fa7ea05932d64d9705b1fb044
I20260810 00:51:27.980659 12369 ts_manager.cc:194] Registered new tserver with Master: e3c5dce1cb384204acc347d4b3e2a29d (127.10.97.65:39961)
I20260810 00:51:27.982091 12369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:58387
WARNING: no policy specified for kudu/127.10.97.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
I20260810 00:51:28.018433 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:43905
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:35019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:28.319336 12686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:28.319569 12686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:28.319705 12686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:28.326520 12686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:28.326612 12686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:28.326745 12686 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 00:51:28.326843 12686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:51:28.335783 12686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:35019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.97.126:43905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:28.336798 12686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:28.337941 12686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:28.349673 12693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:28.349730 12695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:28.349952 12692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:28.350476 12686 server_base.cc:1034] running on GCE node
I20260810 00:51:28.351032 12686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:28.352133 12686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:28.353343 12686 hybrid_clock.cc:648] HybridClock initialized: now 1786323088353275 us; error 45 us; skew 500 ppm
I20260810 00:51:28.353627 12686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:28 dist-test-slave-jlzn krb5kdc[12308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323088, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:28.356712 12686 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:51:28.358035 12686 webserver.cc:460] Webserver started at http://127.10.97.66:41191/ using document root <none> and password file <none>
I20260810 00:51:28.358647 12686 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:28.358739 12686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:28.359038 12686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:28.361666 12686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "f5addb1ac9954a768ea5015e913f13e2"
format_stamp: "Formatted at 2026-08-10 00:51:28 on dist-test-slave-jlzn"
server_key: "38107bf760cdeca880c0100edf3a12ce"
server_key_iv: "0010c93a086da8395effca0a864ab59b"
server_key_version: "encryptionkey@0"
I20260810 00:51:28.362414 12686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "f5addb1ac9954a768ea5015e913f13e2"
format_stamp: "Formatted at 2026-08-10 00:51:28 on dist-test-slave-jlzn"
server_key: "38107bf760cdeca880c0100edf3a12ce"
server_key_iv: "0010c93a086da8395effca0a864ab59b"
server_key_version: "encryptionkey@0"
I20260810 00:51:28.366999 12686 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260810 00:51:28.370204 12705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:28.371397 12686 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:51:28.371552 12686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "f5addb1ac9954a768ea5015e913f13e2"
format_stamp: "Formatted at 2026-08-10 00:51:28 on dist-test-slave-jlzn"
server_key: "38107bf760cdeca880c0100edf3a12ce"
server_key_iv: "0010c93a086da8395effca0a864ab59b"
server_key_version: "encryptionkey@0"
I20260810 00:51:28.371737 12686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:28.399119 12686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:28.402663 12686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:28.402892 12686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:28.405405 12686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:28.405501 12686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:28.405650 12686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:28.405715 12686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 00:51:28 dist-test-slave-jlzn krb5kdc[12308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323088, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:28.428560 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:28.405688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:57173 (local address 127.10.97.126:43905)
0810 00:51:28.405849 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:28.405853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:28.407235 (+ 1382us) server_negotiation.cc:407] Connection header received
0810 00:51:28.409235 (+ 2000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:28.409240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:28.409316 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:28.409400 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:28.411383 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:28.412543 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:28.415506 (+ 2963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:28.416199 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:28.416321 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:28.420698 (+ 4377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:28.420706 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:28.420711 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:28.420755 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:28.424382 (+ 3627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:28.425422 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:28.425428 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:28.425432 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:28.425590 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:28.426210 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:28.426215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:28.426218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:28.426620 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:28.427920 (+ 1300us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:28.428047 (+ 127us) server_negotiation.cc:299] Negotiation successful
0810 00:51:28.428263 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 00:51:28.429560 12716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:28.406307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43905 (local address 127.10.97.66:57173)
0810 00:51:28.407085 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:28.407124 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:28.408989 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:28.409663 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:28.409674 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:28.410375 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:28.411178 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:28.411200 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:28.412934 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:28.412940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:28.415341 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:28.415358 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:28.416429 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:28.416437 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:28.416712 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:28.417389 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:28.417419 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:28.420441 (+ 3022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:28.424561 (+ 4120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:28.424569 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:28.424583 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:28.425244 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:28.425732 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:28.425741 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:28.425744 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:28.426027 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:28.426789 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:28.426798 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:28.427208 (+ 410us) client_negotiation.cc:769] Sending connection context
0810 00:51:28.427485 (+ 277us) client_negotiation.cc:240] Negotiation successful
0810 00:51:28.428011 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":365,"threads_started":1}
I20260810 00:51:28.454187 12818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:28.439585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43905 (local address 127.10.97.66:49923)
0810 00:51:28.440304 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:28.440331 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:28.440510 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:28.440805 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:28.440809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:28.441457 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:28.441736 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:28.441745 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:28.443077 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:28.443086 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:28.444963 (+ 1877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:28.444982 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:28.445934 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:28.445944 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:28.446144 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:28.446762 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:28.446778 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:28.447738 (+ 960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:28.451270 (+ 3532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:28.451276 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:28.451283 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:28.451968 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:28.452419 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:28.452424 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:28.452432 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:28.452605 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:28.453328 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:28.453334 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:28.453552 (+ 218us) client_negotiation.cc:769] Sending connection context
0810 00:51:28.453670 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 00:51:28.453876 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":427,"threads_started":1}
I20260810 00:51:28.454432 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:28.439527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:49923 (local address 127.10.97.126:43905)
0810 00:51:28.439695 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:28.439701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:28.440417 (+ 716us) server_negotiation.cc:407] Connection header received
0810 00:51:28.440631 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:28.440636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:28.440697 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:28.440769 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:28.441868 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:28.442895 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:28.445126 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:28.445722 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:28.445848 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:28.447917 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:28.447924 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:28.447928 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:28.447957 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:28.451086 (+ 3129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:28.452127 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:28.452133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:28.452137 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:28.452294 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:28.452767 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:28.452773 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:28.452777 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:28.453191 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:28.453364 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:28.453826 (+ 462us) server_negotiation.cc:299] Negotiation successful
0810 00:51:28.454072 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 00:51:28.456408 12686 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:35685
I20260810 00:51:28.456552 12898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:35685 every 8 connection(s)
I20260810 00:51:28.457563 12686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:51:28.459453 12899 heartbeater.cc:347] Connected to a master server at 127.10.97.126:43905
I20260810 00:51:28.461022 12899 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:28.461582 12899 heartbeater.cc:511] Master 127.10.97.126:43905 requested a full tablet report, sending...
I20260810 00:51:28.462522 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 12686
I20260810 00:51:28.462738 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:51:28.462785 12369 ts_manager.cc:194] Registered new tserver with Master: f5addb1ac9954a768ea5015e913f13e2 (127.10.97.66:35685)
I20260810 00:51:28.463308 10629 external_mini_cluster.cc:1428] Setting key 123a51dd4ae7c682aaea3a24f51038e4
I20260810 00:51:28.463838 12369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:57173
WARNING: no policy specified for kudu/127.10.97.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
I20260810 00:51:28.501994 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:43905
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:35019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:28.804209 12908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:28.804461 12908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:28.804562 12908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:28.812067 12908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:28.812179 12908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:28.812286 12908 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 00:51:28.812364 12908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:51:28.821415 12908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:35019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.97.126:43905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:28.822433 12908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:28.823609 12908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:28.835302 12915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:28.835440 12916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:28.835445 12918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:28.836463 12908 server_base.cc:1034] running on GCE node
I20260810 00:51:28.837057 12908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:28.837832 12908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:28.839043 12908 hybrid_clock.cc:648] HybridClock initialized: now 1786323088838983 us; error 31 us; skew 500 ppm
I20260810 00:51:28.839309 12908 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:28 dist-test-slave-jlzn krb5kdc[12308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323088, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:28.842155 12908 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:51:28.843438 12908 webserver.cc:460] Webserver started at http://127.10.97.67:40167/ using document root <none> and password file <none>
I20260810 00:51:28.844100 12908 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:28.844203 12908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:28.844491 12908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:28.847100 12908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "b401ff211e244d0da16bf35a1e68e931"
format_stamp: "Formatted at 2026-08-10 00:51:28 on dist-test-slave-jlzn"
server_key: "755071b879e880e51bfc184a395a0a94"
server_key_iv: "a70c3d4f2f4a6a9bb74bd3a987a84b10"
server_key_version: "encryptionkey@0"
I20260810 00:51:28.847883 12908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "b401ff211e244d0da16bf35a1e68e931"
format_stamp: "Formatted at 2026-08-10 00:51:28 on dist-test-slave-jlzn"
server_key: "755071b879e880e51bfc184a395a0a94"
server_key_iv: "a70c3d4f2f4a6a9bb74bd3a987a84b10"
server_key_version: "encryptionkey@0"
I20260810 00:51:28.852598 12908 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 00:51:28.855700 12929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:28.857002 12908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 00:51:28.857139 12908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "b401ff211e244d0da16bf35a1e68e931"
format_stamp: "Formatted at 2026-08-10 00:51:28 on dist-test-slave-jlzn"
server_key: "755071b879e880e51bfc184a395a0a94"
server_key_iv: "a70c3d4f2f4a6a9bb74bd3a987a84b10"
server_key_version: "encryptionkey@0"
I20260810 00:51:28.857327 12908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:28.881130 12908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:28.884609 12908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:28.884833 12908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:28.887156 12908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:28.887243 12908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:28.887372 12908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:28.887436 12908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 00:51:28 dist-test-slave-jlzn krb5kdc[12308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323088, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:28.909220 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:28.887664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:35273 (local address 127.10.97.126:43905)
0810 00:51:28.887890 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:28.887896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:28.889264 (+ 1368us) server_negotiation.cc:407] Connection header received
0810 00:51:28.891919 (+ 2655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:28.891925 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:28.892015 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:28.892133 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:28.894417 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:28.895384 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:28.898223 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:28.898889 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:28.899025 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:28.902744 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:28.902749 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:28.902753 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:28.902783 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:28.905707 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:28.906604 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:28.906611 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:28.906614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:28.906806 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:28.907295 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:28.907300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:28.907304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:28.907585 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:28.907743 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:28.908643 (+ 900us) server_negotiation.cc:299] Negotiation successful
0810 00:51:28.908872 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 00:51:28.910292 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:28.888246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43905 (local address 127.10.97.67:35273)
0810 00:51:28.889084 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:28.889141 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:28.891572 (+ 2431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:28.892410 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:28.892420 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:28.893248 (+ 828us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:28.894201 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:28.894227 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:28.895573 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:28.895579 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:28.898014 (+ 2435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:28.898030 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:28.899126 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:28.899134 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:28.899414 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:28.900102 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:28.900125 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:28.902517 (+ 2392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:28.905828 (+ 3311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:28.905843 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:28.905857 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:28.906440 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:28.906948 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:28.906953 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:28.906957 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:28.907166 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:28.907851 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:28.907858 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:28.908267 (+ 409us) client_negotiation.cc:769] Sending connection context
0810 00:51:28.908508 (+ 241us) client_negotiation.cc:240] Negotiation successful
0810 00:51:28.909007 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":330,"threads_started":1}
I20260810 00:51:28.923189 12908 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:37809
I20260810 00:51:28.923241 13130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:37809 every 8 connection(s)
I20260810 00:51:28.924458 12908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:51:28.926615 13132 heartbeater.cc:347] Connected to a master server at 127.10.97.126:43905
I20260810 00:51:28.926824 13132 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:28.927433 13132 heartbeater.cc:511] Master 127.10.97.126:43905 requested a full tablet report, sending...
I20260810 00:51:28.928699 12369 ts_manager.cc:194] Registered new tserver with Master: b401ff211e244d0da16bf35a1e68e931 (127.10.97.67:37809)
I20260810 00:51:28.929363 13097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:28.916339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43905 (local address 127.10.97.67:34819)
0810 00:51:28.917011 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:28.917034 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:28.917175 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:28.917526 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:28.917534 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:28.918151 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:28.918514 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:28.918532 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:28.919900 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:28.919905 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:28.921386 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:28.921398 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:28.922349 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:28.922358 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:28.922530 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:28.923058 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:28.923077 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:28.923871 (+ 794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:28.926897 (+ 3026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:28.926902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:28.926906 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:28.927299 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:28.927683 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:28.927688 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:28.927692 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:28.927852 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:28.928391 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:28.928395 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:28.928553 (+ 158us) client_negotiation.cc:769] Sending connection context
0810 00:51:28.928679 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 00:51:28.928950 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":369,"threads_started":1}
I20260810 00:51:28.929386 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:28.916216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:34819 (local address 127.10.97.126:43905)
0810 00:51:28.916383 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:28.916389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:28.917120 (+ 731us) server_negotiation.cc:407] Connection header received
0810 00:51:28.917308 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:28.917314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:28.917387 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:28.917485 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:28.918686 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:28.919746 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:28.921503 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:28.922138 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:28.922268 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:28.924033 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:28.924039 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:28.924042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:28.924070 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:28.926788 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:28.927408 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:28.927414 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:28.927417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:28.927573 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:28.927950 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:28.927956 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:28.927960 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:28.928286 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:28.928413 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:28.928818 (+ 405us) server_negotiation.cc:299] Negotiation successful
0810 00:51:28.929021 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 00:51:28.929536 12369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:35273
I20260810 00:51:28.934692 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 12908
I20260810 00:51:28.934864 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:51:28.935266 10629 external_mini_cluster.cc:1428] Setting key 5f7a5b9253c2aacf31d63260137020be
I20260810 00:51:28.937777 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 00:51:28 dist-test-slave-jlzn krb5kdc[12308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323088, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 00:51:28 dist-test-slave-jlzn krb5kdc[12308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323088, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:28.984978 12680 heartbeater.cc:503] Master 127.10.97.126:43905 was elected leader, sending a full tablet report...
I20260810 00:51:28.988029 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:28.974185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39254 (local address 127.10.97.126:43905)
0810 00:51:28.974355 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:28.974361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:28.974881 (+ 520us) server_negotiation.cc:407] Connection header received
0810 00:51:28.975071 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:28.975075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:28.975157 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:28.975230 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:28.976898 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:28.977620 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:28.979274 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:28.979664 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:28.979753 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:28.983219 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:28.983224 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:28.983227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:28.983259 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:28.985823 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:28.986396 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:28.986401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:28.986404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:28.986550 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:28.986852 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:28.986855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:28.986857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:28.987159 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:28.987332 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:28.987554 (+ 222us) server_negotiation.cc:299] Negotiation successful
0810 00:51:28.987706 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 00:51:28.991367 12369 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:51:28.997962 12369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:51:29.018028 13151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.008757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43988 (local address 127.10.97.65:39961)
0810 00:51:29.010578 (+ 1821us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:29.010586 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:29.010612 (+ 26us) server_negotiation.cc:407] Connection header received
0810 00:51:29.010707 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:29.010714 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:29.010905 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:29.011069 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:29.011850 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.013323 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.015504 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.016925 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.017062 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.017256 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:29.017587 (+ 331us) server_negotiation.cc:299] Negotiation successful
0810 00:51:29.017748 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1677,"thread_start_us":1541,"threads_started":1}
I20260810 00:51:29.018908 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.008641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:39961 (local address 127.0.0.1:43988)
0810 00:51:29.009561 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:29.009597 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:29.009757 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:29.011100 (+ 1343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:29.011107 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:29.011136 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:29.011609 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.011631 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.013469 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.013477 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.015350 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.015368 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.017109 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.017116 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.017350 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.017401 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 00:51:29.017526 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 00:51:29.017698 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":3,"thread_start_us":569,"threads_started":1}
I20260810 00:51:29.019654 13155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.009428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:37809 (local address 127.0.0.1:51086)
0810 00:51:29.011769 (+ 2341us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:29.011784 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:29.011907 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:29.012743 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:29.012747 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:29.012771 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:29.013493 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.013503 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.015386 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.015390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.016845 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.016857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.018873 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.018881 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.019059 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.019092 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 00:51:29.019171 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 00:51:29.019288 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2208,"mutex_wait_us":35,"thread_start_us":2006,"threads_started":1}
I20260810 00:51:29.019822 13153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.008900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:35685 (local address 127.0.0.1:36506)
0810 00:51:29.010076 (+ 1176us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:29.010088 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:29.010176 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:29.012777 (+ 2601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:29.012782 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:29.012807 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:29.013203 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.013222 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.014973 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.014978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.017187 (+ 2209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.017202 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.018779 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.018786 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.018960 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.019254 (+ 294us) client_negotiation.cc:769] Sending connection context
0810 00:51:29.019318 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 00:51:29.019434 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1077,"mutex_wait_us":62,"thread_start_us":1386,"threads_started":1}
I20260810 00:51:29.019989 13156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.010056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51086 (local address 127.10.97.67:37809)
0810 00:51:29.010763 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:29.010770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:29.012071 (+ 1301us) server_negotiation.cc:407] Connection header received
0810 00:51:29.012186 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:29.012194 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:29.012391 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:29.012563 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:29.013712 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.015225 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.016980 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.018367 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.018486 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.018715 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:29.019234 (+ 519us) server_negotiation.cc:299] Negotiation successful
0810 00:51:29.019387 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":362,"threads_started":1}
I20260810 00:51:29.020602 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.009416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36506 (local address 127.10.97.66:35685)
0810 00:51:29.012189 (+ 2773us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:29.012196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:29.012232 (+ 36us) server_negotiation.cc:407] Connection header received
0810 00:51:29.012326 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:29.012333 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:29.012561 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:29.012724 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:29.013363 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.014817 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.017358 (+ 2541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.018539 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.020035 (+ 1496us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.020235 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:29.020299 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 00:51:29.020418 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2619,"thread_start_us":2907,"threads_started":1}
I20260810 00:51:29.022492 12566 tablet_service.cc:1467] Processing CreateTablet for tablet 79e89cea2c4b4276bde2f3d85b9cda56 (DEFAULT_TABLE table=test-table [id=dec6a4ac084d4fc4a9c683f91dc2a55b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:29.023309 13017 tablet_service.cc:1467] Processing CreateTablet for tablet 79e89cea2c4b4276bde2f3d85b9cda56 (DEFAULT_TABLE table=test-table [id=dec6a4ac084d4fc4a9c683f91dc2a55b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:29.023957 12566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79e89cea2c4b4276bde2f3d85b9cda56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:29.023689 12779 tablet_service.cc:1467] Processing CreateTablet for tablet 79e89cea2c4b4276bde2f3d85b9cda56 (DEFAULT_TABLE table=test-table [id=dec6a4ac084d4fc4a9c683f91dc2a55b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:29.024458 13017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79e89cea2c4b4276bde2f3d85b9cda56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:29.024968 12779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79e89cea2c4b4276bde2f3d85b9cda56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:29.035565 13157 tablet_bootstrap.cc:492] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931: Bootstrap starting.
I20260810 00:51:29.037439 13158 tablet_bootstrap.cc:492] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d: Bootstrap starting.
I20260810 00:51:29.039023 13157 tablet_bootstrap.cc:654] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:29.039770 13159 tablet_bootstrap.cc:492] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2: Bootstrap starting.
I20260810 00:51:29.040617 13157 log.cc:824] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:29.041452 13158 tablet_bootstrap.cc:654] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:29.042524 13158 log.cc:824] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:29.042702 13159 tablet_bootstrap.cc:654] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:29.043512 13157 tablet_bootstrap.cc:492] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931: No bootstrap required, opened a new log
I20260810 00:51:29.043821 13157 ts_tablet_manager.cc:1397] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260810 00:51:29.043870 13159 log.cc:824] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:29.045400 13158 tablet_bootstrap.cc:492] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d: No bootstrap required, opened a new log
I20260810 00:51:29.045657 13158 ts_tablet_manager.cc:1397] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260810 00:51:29.046109 13159 tablet_bootstrap.cc:492] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2: No bootstrap required, opened a new log
I20260810 00:51:29.046335 13159 ts_tablet_manager.cc:1397] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260810 00:51:29.053519 13159 raft_consensus.cc:348] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.053833 13159 raft_consensus.cc:374] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:29.053928 13159 raft_consensus.cc:729] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5addb1ac9954a768ea5015e913f13e2, State: Initialized, Role: FOLLOWER
I20260810 00:51:29.053905 13157 raft_consensus.cc:348] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.054296 13157 raft_consensus.cc:374] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:29.054394 13157 raft_consensus.cc:729] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b401ff211e244d0da16bf35a1e68e931, State: Initialized, Role: FOLLOWER
I20260810 00:51:29.054481 13159 consensus_queue.cc:260] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.054916 13157 consensus_queue.cc:260] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.055567 12899 heartbeater.cc:503] Master 127.10.97.126:43905 was elected leader, sending a full tablet report...
I20260810 00:51:29.056032 13159 ts_tablet_manager.cc:1428] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260810 00:51:29.056514 13158 raft_consensus.cc:348] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.056880 13158 raft_consensus.cc:374] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:29.056981 13158 raft_consensus.cc:729] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c5dce1cb384204acc347d4b3e2a29d, State: Initialized, Role: FOLLOWER
I20260810 00:51:29.057497 13158 consensus_queue.cc:260] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.060827 13158 ts_tablet_manager.cc:1428] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260810 00:51:29.061751 13132 heartbeater.cc:503] Master 127.10.97.126:43905 was elected leader, sending a full tablet report...
I20260810 00:51:29.062331 13157 ts_tablet_manager.cc:1428] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931: Time spent starting tablet: real 0.018s user 0.009s sys 0.008s
I20260810 00:51:29.111792 13163 raft_consensus.cc:482] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:51:29.112121 13163 raft_consensus.cc:504] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.113725 13163 leader_election.cc:288] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3c5dce1cb384204acc347d4b3e2a29d (127.10.97.65:39961), b401ff211e244d0da16bf35a1e68e931 (127.10.97.67:37809)
I20260810 00:51:29.121627 13151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.114121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:46103 (local address 127.10.97.65:39961)
0810 00:51:29.114311 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:29.114317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:29.114959 (+ 642us) server_negotiation.cc:407] Connection header received
0810 00:51:29.115177 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:29.115180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:29.115238 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:29.115325 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:29.115926 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.116946 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.119420 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.120639 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.120799 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.120871 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:29.121265 (+ 394us) server_negotiation.cc:299] Negotiation successful
0810 00:51:29.121388 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 00:51:29.121626 13156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.114206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:43625 (local address 127.10.97.67:37809)
0810 00:51:29.114370 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:29.114375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:29.115288 (+ 913us) server_negotiation.cc:407] Connection header received
0810 00:51:29.115420 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:29.115424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:29.115491 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:29.115555 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:29.116203 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.117190 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.119491 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.120668 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.120808 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.120871 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:29.121265 (+ 394us) server_negotiation.cc:299] Negotiation successful
0810 00:51:29.121388 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 00:51:29.121877 13167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.113915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:39961 (local address 127.10.97.66:46103)
0810 00:51:29.114847 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:29.114865 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:29.115016 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:29.115361 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:29.115365 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:29.115389 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:29.115772 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.115784 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.117068 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.117072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.119264 (+ 2192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.119279 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.120903 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.120909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.121096 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.121154 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 00:51:29.121237 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 00:51:29.121385 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":818,"mutex_wait_us":53,"thread_start_us":645,"threads_started":1}
I20260810 00:51:29.121848 13168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.114212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:37809 (local address 127.10.97.66:43625)
0810 00:51:29.115181 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:29.115196 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:29.115312 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:29.115604 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:29.115609 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:29.115632 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:29.116058 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.116067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.117305 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.117310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.119353 (+ 2043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.119364 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.120883 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.120892 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.121103 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.121143 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 00:51:29.121246 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 00:51:29.121385 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":871,"mutex_wait_us":81,"thread_start_us":678,"threads_started":1}
I20260810 00:51:29.122831 12607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79e89cea2c4b4276bde2f3d85b9cda56" candidate_uuid: "f5addb1ac9954a768ea5015e913f13e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" is_pre_election: true
I20260810 00:51:29.123032 13050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79e89cea2c4b4276bde2f3d85b9cda56" candidate_uuid: "f5addb1ac9954a768ea5015e913f13e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b401ff211e244d0da16bf35a1e68e931" is_pre_election: true
I20260810 00:51:29.123394 12607 raft_consensus.cc:2454] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5addb1ac9954a768ea5015e913f13e2 in term 0.
I20260810 00:51:29.123548 13050 raft_consensus.cc:2454] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5addb1ac9954a768ea5015e913f13e2 in term 0.
I20260810 00:51:29.124074 12708 leader_election.cc:302] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e3c5dce1cb384204acc347d4b3e2a29d, f5addb1ac9954a768ea5015e913f13e2; no voters:
I20260810 00:51:29.124442 13163 raft_consensus.cc:2781] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:51:29.124611 13163 raft_consensus.cc:482] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:51:29.124717 13163 raft_consensus.cc:3037] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:29.126829 13163 raft_consensus.cc:504] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.127357 13163 leader_election.cc:288] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [CANDIDATE]: Term 1 election: Requested vote from peers e3c5dce1cb384204acc347d4b3e2a29d (127.10.97.65:39961), b401ff211e244d0da16bf35a1e68e931 (127.10.97.67:37809)
I20260810 00:51:29.127722 12607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79e89cea2c4b4276bde2f3d85b9cda56" candidate_uuid: "f5addb1ac9954a768ea5015e913f13e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c5dce1cb384204acc347d4b3e2a29d"
I20260810 00:51:29.127833 13050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79e89cea2c4b4276bde2f3d85b9cda56" candidate_uuid: "f5addb1ac9954a768ea5015e913f13e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b401ff211e244d0da16bf35a1e68e931"
I20260810 00:51:29.127950 12607 raft_consensus.cc:3037] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:29.128017 13050 raft_consensus.cc:3037] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:29.131049 12607 raft_consensus.cc:2454] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5addb1ac9954a768ea5015e913f13e2 in term 1.
I20260810 00:51:29.131049 13050 raft_consensus.cc:2454] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5addb1ac9954a768ea5015e913f13e2 in term 1.
I20260810 00:51:29.131424 12708 leader_election.cc:302] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: b401ff211e244d0da16bf35a1e68e931, e3c5dce1cb384204acc347d4b3e2a29d, f5addb1ac9954a768ea5015e913f13e2; no voters:
I20260810 00:51:29.131753 13163 raft_consensus.cc:2781] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:29.132175 13163 raft_consensus.cc:686] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [term 1 LEADER]: Becoming Leader. State: Replica: f5addb1ac9954a768ea5015e913f13e2, State: Running, Role: LEADER
I20260810 00:51:29.132627 13163 consensus_queue.cc:237] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.136583 12369 catalog_manager.cc:5495] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5addb1ac9954a768ea5015e913f13e2 (127.10.97.66). New cstate: current_term: 1 leader_uuid: "f5addb1ac9954a768ea5015e913f13e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } health_report { overall_health: UNKNOWN } } }
W20260810 00:51:29.176012 13134 tablet.cc:2369] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:51:29.182281 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.176964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36508 (local address 127.10.97.66:35685)
0810 00:51:29.177145 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:29.177151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:29.177214 (+ 63us) server_negotiation.cc:407] Connection header received
0810 00:51:29.177548 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:29.177551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:29.177629 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:29.177740 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:51:29.178083 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.178769 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.180129 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.180547 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.180654 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.181135 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:51:29.181701 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:51:29.181825 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:29.181953 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 00:51:29.182030 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 00:51:29.190932 13050 raft_consensus.cc:1261] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931 [term 1 FOLLOWER]: Refusing update from remote peer f5addb1ac9954a768ea5015e913f13e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:51:29.192032 13170 consensus_queue.cc:1035] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:51:29.192704 12607 raft_consensus.cc:1261] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d [term 1 FOLLOWER]: Refusing update from remote peer f5addb1ac9954a768ea5015e913f13e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:51:29.193423 13170 consensus_queue.cc:1035] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:51:29.204201 13176 mvcc.cc:205] Tried to move back new op lower bound from 7316779373314813952 to 7316779373090721792. Current Snapshot: MvccSnapshot[applied={T|T < 7316779373314813952}]
I20260810 00:51:29.205622 13178 mvcc.cc:205] Tried to move back new op lower bound from 7316779373314813952 to 7316779373090721792. Current Snapshot: MvccSnapshot[applied={T|T < 7316779373314813952}]
I20260810 00:51:29.222594 12369 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:51:29.222883 12369 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:51:29.226598 12369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd: Sending DeleteTablet for 3 replicas of tablet 79e89cea2c4b4276bde2f3d85b9cda56
I20260810 00:51:29.227767 12566 tablet_service.cc:1514] Processing DeleteTablet for tablet 79e89cea2c4b4276bde2f3d85b9cda56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:29 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:43988
I20260810 00:51:29.227872 12779 tablet_service.cc:1514] Processing DeleteTablet for tablet 79e89cea2c4b4276bde2f3d85b9cda56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:29 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:36506
I20260810 00:51:29.229910 13192 tablet_replica.cc:330] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d: stopping tablet replica
I20260810 00:51:29.229854 13017 tablet_service.cc:1514] Processing DeleteTablet for tablet 79e89cea2c4b4276bde2f3d85b9cda56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:29 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:51086
I20260810 00:51:29.230434 13192 raft_consensus.cc:2229] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:51:29.230969 13192 raft_consensus.cc:2258] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:29.231752 12369 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:51:29.232414 12369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:51:29.238617 13194 tablet_replica.cc:330] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931: stopping tablet replica
I20260810 00:51:29.239032 13194 raft_consensus.cc:2229] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:51:29.239120 13192 ts_tablet_manager.cc:1905] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:29.240399 12778 tablet_service.cc:1467] Processing CreateTablet for tablet c0d9a68fda204278abc6a94accffde9b (DEFAULT_TABLE table=test-table [id=9d83d80dfb6b4be48c90cd733b72b31f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:29.241041 12778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0d9a68fda204278abc6a94accffde9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:29.241012 12566 tablet_service.cc:1467] Processing CreateTablet for tablet c0d9a68fda204278abc6a94accffde9b (DEFAULT_TABLE table=test-table [id=9d83d80dfb6b4be48c90cd733b72b31f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:29.241233 13194 raft_consensus.cc:2258] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:29.241465 12566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0d9a68fda204278abc6a94accffde9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:29.242192 13195 tablet_replica.cc:330] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2: stopping tablet replica
I20260810 00:51:29.242568 13195 raft_consensus.cc:2229] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:51:29.243346 13195 raft_consensus.cc:2258] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:29.245445 13194 ts_tablet_manager.cc:1905] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:29.245476 13017 tablet_service.cc:1467] Processing CreateTablet for tablet c0d9a68fda204278abc6a94accffde9b (DEFAULT_TABLE table=test-table [id=9d83d80dfb6b4be48c90cd733b72b31f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:29.245992 13017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0d9a68fda204278abc6a94accffde9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:29.246951 13192 ts_tablet_manager.cc:1918] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:51:29.247227 13192 log.cc:1196] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/79e89cea2c4b4276bde2f3d85b9cda56
I20260810 00:51:29.247485 13159 tablet_bootstrap.cc:492] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2: Bootstrap starting.
I20260810 00:51:29.247911 13192 ts_tablet_manager.cc:1939] T 79e89cea2c4b4276bde2f3d85b9cda56 P e3c5dce1cb384204acc347d4b3e2a29d: Deleting consensus metadata
I20260810 00:51:29.249234 13159 tablet_bootstrap.cc:654] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:29.250432 12350 catalog_manager.cc:4841] TS e3c5dce1cb384204acc347d4b3e2a29d (127.10.97.65:39961): tablet 79e89cea2c4b4276bde2f3d85b9cda56 (table test-table [id=dec6a4ac084d4fc4a9c683f91dc2a55b]) successfully deleted
I20260810 00:51:29.253577 13195 ts_tablet_manager.cc:1905] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:29.258148 13158 tablet_bootstrap.cc:492] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d: Bootstrap starting.
I20260810 00:51:29.259640 13158 tablet_bootstrap.cc:654] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:29.259876 13157 tablet_bootstrap.cc:492] T c0d9a68fda204278abc6a94accffde9b P b401ff211e244d0da16bf35a1e68e931: Bootstrap starting.
I20260810 00:51:29.260582 13194 ts_tablet_manager.cc:1918] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:51:29.260766 13194 log.cc:1196] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/79e89cea2c4b4276bde2f3d85b9cda56
I20260810 00:51:29.261452 13157 tablet_bootstrap.cc:654] T c0d9a68fda204278abc6a94accffde9b P b401ff211e244d0da16bf35a1e68e931: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:29.262240 13159 tablet_bootstrap.cc:492] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2: No bootstrap required, opened a new log
I20260810 00:51:29.262406 13159 ts_tablet_manager.cc:1397] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260810 00:51:29.263263 13159 raft_consensus.cc:348] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.263504 13159 raft_consensus.cc:374] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:29.263607 13159 raft_consensus.cc:729] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5addb1ac9954a768ea5015e913f13e2, State: Initialized, Role: FOLLOWER
I20260810 00:51:29.263921 13159 consensus_queue.cc:260] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.264654 13159 ts_tablet_manager.cc:1428] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 00:51:29.265522 13194 ts_tablet_manager.cc:1939] T 79e89cea2c4b4276bde2f3d85b9cda56 P b401ff211e244d0da16bf35a1e68e931: Deleting consensus metadata
I20260810 00:51:29.271450 12348 catalog_manager.cc:4841] TS b401ff211e244d0da16bf35a1e68e931 (127.10.97.67:37809): tablet 79e89cea2c4b4276bde2f3d85b9cda56 (table test-table [id=dec6a4ac084d4fc4a9c683f91dc2a55b]) successfully deleted
I20260810 00:51:29.271854 13195 ts_tablet_manager.cc:1918] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:51:29.272224 13195 log.cc:1196] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/79e89cea2c4b4276bde2f3d85b9cda56
I20260810 00:51:29.273432 13195 ts_tablet_manager.cc:1939] T 79e89cea2c4b4276bde2f3d85b9cda56 P f5addb1ac9954a768ea5015e913f13e2: Deleting consensus metadata
I20260810 00:51:29.277319 12351 catalog_manager.cc:4841] TS f5addb1ac9954a768ea5015e913f13e2 (127.10.97.66:35685): tablet 79e89cea2c4b4276bde2f3d85b9cda56 (table test-table [id=dec6a4ac084d4fc4a9c683f91dc2a55b]) successfully deleted
I20260810 00:51:29.278820 13157 tablet_bootstrap.cc:492] T c0d9a68fda204278abc6a94accffde9b P b401ff211e244d0da16bf35a1e68e931: No bootstrap required, opened a new log
I20260810 00:51:29.279068 13158 tablet_bootstrap.cc:492] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d: No bootstrap required, opened a new log
I20260810 00:51:29.279196 13157 ts_tablet_manager.cc:1397] T c0d9a68fda204278abc6a94accffde9b P b401ff211e244d0da16bf35a1e68e931: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.007s
I20260810 00:51:29.279419 13158 ts_tablet_manager.cc:1397] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.000s
I20260810 00:51:29.281328 13157 raft_consensus.cc:348] T c0d9a68fda204278abc6a94accffde9b P b401ff211e244d0da16bf35a1e68e931 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.281432 13158 raft_consensus.cc:348] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.282012 13157 raft_consensus.cc:374] T c0d9a68fda204278abc6a94accffde9b P b401ff211e244d0da16bf35a1e68e931 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:29.282022 13158 raft_consensus.cc:374] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:29.282249 13158 raft_consensus.cc:729] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c5dce1cb384204acc347d4b3e2a29d, State: Initialized, Role: FOLLOWER
I20260810 00:51:29.282249 13157 raft_consensus.cc:729] T c0d9a68fda204278abc6a94accffde9b P b401ff211e244d0da16bf35a1e68e931 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b401ff211e244d0da16bf35a1e68e931, State: Initialized, Role: FOLLOWER
I20260810 00:51:29.282886 13158 consensus_queue.cc:260] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.282871 13157 consensus_queue.cc:260] T c0d9a68fda204278abc6a94accffde9b P b401ff211e244d0da16bf35a1e68e931 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.284268 13158 ts_tablet_manager.cc:1428] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260810 00:51:29.285777 13157 ts_tablet_manager.cc:1428] T c0d9a68fda204278abc6a94accffde9b P b401ff211e244d0da16bf35a1e68e931: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 00:51:29.305482 13166 raft_consensus.cc:482] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:51:29.305749 13166 raft_consensus.cc:504] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.309374 13166 leader_election.cc:288] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5addb1ac9954a768ea5015e913f13e2 (127.10.97.66:35685), b401ff211e244d0da16bf35a1e68e931 (127.10.97.67:37809)
I20260810 00:51:29.325199 13203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.310300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:37809 (local address 127.10.97.65:59995)
0810 00:51:29.312835 (+ 2535us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:29.312903 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:29.313204 (+ 301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:29.314442 (+ 1238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:29.314449 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:29.314509 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:29.315241 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.315268 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.317744 (+ 2476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.317755 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.320485 (+ 2730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.320514 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.323200 (+ 2686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.323208 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.323922 (+ 714us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.323987 (+ 65us) client_negotiation.cc:769] Sending connection context
0810 00:51:29.324184 (+ 197us) client_negotiation.cc:240] Negotiation successful
0810 00:51:29.324446 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2253,"mutex_wait_us":73,"spinlock_wait_cycles":9472,"thread_start_us":1108,"threads_started":1}
I20260810 00:51:29.325385 13156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.310250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:59995 (local address 127.10.97.67:37809)
0810 00:51:29.310731 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:29.310751 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:29.313084 (+ 2333us) server_negotiation.cc:407] Connection header received
0810 00:51:29.313678 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:29.313685 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:29.313910 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:29.314063 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:29.315678 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.317513 (+ 1835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.320716 (+ 3203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.322751 (+ 2035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.323102 (+ 351us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.323298 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:29.324344 (+ 1046us) server_negotiation.cc:299] Negotiation successful
0810 00:51:29.324770 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260810 00:51:29.326690 13050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0d9a68fda204278abc6a94accffde9b" candidate_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b401ff211e244d0da16bf35a1e68e931" is_pre_election: true
I20260810 00:51:29.326948 13201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.310065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:35685 (local address 127.10.97.65:45479)
0810 00:51:29.311678 (+ 1613us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:29.311722 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:29.312046 (+ 324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:29.313086 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:29.313104 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:29.313195 (+ 91us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:29.314077 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.314112 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.316969 (+ 2857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.316987 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.321564 (+ 4577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.321594 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.325284 (+ 3690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.325304 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.325702 (+ 398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.325793 (+ 91us) client_negotiation.cc:769] Sending connection context
0810 00:51:29.325953 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 00:51:29.326175 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1298,"mutex_wait_us":152,"spinlock_wait_cycles":10624,"thread_start_us":2037,"threads_started":1}
I20260810 00:51:29.327198 13050 raft_consensus.cc:2454] T c0d9a68fda204278abc6a94accffde9b P b401ff211e244d0da16bf35a1e68e931 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3c5dce1cb384204acc347d4b3e2a29d in term 0.
I20260810 00:51:29.328099 12499 leader_election.cc:302] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b401ff211e244d0da16bf35a1e68e931, e3c5dce1cb384204acc347d4b3e2a29d; no voters:
I20260810 00:51:29.328655 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.311031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:45479 (local address 127.10.97.66:35685)
0810 00:51:29.311391 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:29.311411 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:29.311892 (+ 481us) server_negotiation.cc:407] Connection header received
0810 00:51:29.312328 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:29.312336 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:29.312572 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:29.312658 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:29.314444 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.316701 (+ 2257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.321841 (+ 5140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.324659 (+ 2818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.324937 (+ 278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.325010 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:29.327717 (+ 2707us) server_negotiation.cc:299] Negotiation successful
0810 00:51:29.328007 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260810 00:51:29.328851 13166 raft_consensus.cc:2781] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:51:29.329070 13166 raft_consensus.cc:482] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:51:29.329209 13166 raft_consensus.cc:3037] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:29.330260 12825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0d9a68fda204278abc6a94accffde9b" candidate_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5addb1ac9954a768ea5015e913f13e2" is_pre_election: true
I20260810 00:51:29.331130 12825 raft_consensus.cc:2454] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3c5dce1cb384204acc347d4b3e2a29d in term 0.
I20260810 00:51:29.334242 13166 raft_consensus.cc:504] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.335305 13166 leader_election.cc:288] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [CANDIDATE]: Term 1 election: Requested vote from peers f5addb1ac9954a768ea5015e913f13e2 (127.10.97.66:35685), b401ff211e244d0da16bf35a1e68e931 (127.10.97.67:37809)
I20260810 00:51:29.336103 13050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0d9a68fda204278abc6a94accffde9b" candidate_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b401ff211e244d0da16bf35a1e68e931"
I20260810 00:51:29.336030 12825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0d9a68fda204278abc6a94accffde9b" candidate_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5addb1ac9954a768ea5015e913f13e2"
I20260810 00:51:29.336381 13050 raft_consensus.cc:3037] T c0d9a68fda204278abc6a94accffde9b P b401ff211e244d0da16bf35a1e68e931 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:29.336422 12825 raft_consensus.cc:3037] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:29.340077 13050 raft_consensus.cc:2454] T c0d9a68fda204278abc6a94accffde9b P b401ff211e244d0da16bf35a1e68e931 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3c5dce1cb384204acc347d4b3e2a29d in term 1.
I20260810 00:51:29.340642 12499 leader_election.cc:302] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b401ff211e244d0da16bf35a1e68e931, e3c5dce1cb384204acc347d4b3e2a29d; no voters:
I20260810 00:51:29.341055 13166 raft_consensus.cc:2781] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:29.341801 13166 raft_consensus.cc:686] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [term 1 LEADER]: Becoming Leader. State: Replica: e3c5dce1cb384204acc347d4b3e2a29d, State: Running, Role: LEADER
I20260810 00:51:29.342363 13166 consensus_queue.cc:237] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.343369 12825 raft_consensus.cc:2454] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3c5dce1cb384204acc347d4b3e2a29d in term 1.
I20260810 00:51:29.348239 12367 catalog_manager.cc:5495] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d reported cstate change: term changed from 0 to 1, leader changed from <none> to e3c5dce1cb384204acc347d4b3e2a29d (127.10.97.65). New cstate: current_term: 1 leader_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } health_report { overall_health: UNKNOWN } } }
Aug 10 00:51:29 dist-test-slave-jlzn krb5kdc[12308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323087, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:29.440081 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.396843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43905 (local address 127.0.0.1:39262)
0810 00:51:29.398086 (+ 1243us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:29.398120 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:29.398373 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:29.398936 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:29.398942 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:29.400789 (+ 1847us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:29.401477 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.401503 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.403104 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.403110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.405932 (+ 2822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.405959 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.407186 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.407215 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.407551 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.413256 (+ 5705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:29.413322 (+ 66us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:29.420079 (+ 6757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:29.431076 (+ 10997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:29.431111 (+ 35us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:29.431131 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:29.432640 (+ 1509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:29.434254 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:29.434260 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:29.434264 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:29.434644 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:29.436255 (+ 1611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:29.436274 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:29.436701 (+ 427us) client_negotiation.cc:769] Sending connection context
0810 00:51:29.437421 (+ 720us) client_negotiation.cc:240] Negotiation successful
0810 00:51:29.438962 (+ 1541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":932,"thread_start_us":656,"threads_started":1}
I20260810 00:51:29.441926 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.396779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39262 (local address 127.10.97.126:43905)
0810 00:51:29.397095 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:29.397114 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:29.398260 (+ 1146us) server_negotiation.cc:407] Connection header received
0810 00:51:29.398568 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:29.398574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:29.398679 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:29.398820 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:29.401700 (+ 2880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.402901 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.406203 (+ 3302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.406880 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.407041 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.420275 (+ 13234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:29.420282 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:29.420287 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:29.420321 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:29.430224 (+ 9903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:29.433379 (+ 3155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:29.433457 (+ 78us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:29.433462 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:29.433861 (+ 399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:29.434913 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:29.434918 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:29.434922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:29.435775 (+ 853us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:29.436116 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:29.440488 (+ 4372us) server_negotiation.cc:299] Negotiation successful
0810 00:51:29.441232 (+ 744us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260810 00:51:29.451329 12367 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:39262:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 00:51:29.453740 12367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 00:51:29.462234 12901 tablet.cc:2369] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:51:29.468453 12566 tablet_service.cc:1467] Processing CreateTablet for tablet 7ad9722d782e420180336fe51de4f511 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3b7825bba42942e888ea189b445076a7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 00:51:29.469522 12566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ad9722d782e420180336fe51de4f511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:29.469352 12778 tablet_service.cc:1467] Processing CreateTablet for tablet 7ad9722d782e420180336fe51de4f511 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3b7825bba42942e888ea189b445076a7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 00:51:29.470883 12778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ad9722d782e420180336fe51de4f511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:29.471055 13017 tablet_service.cc:1467] Processing CreateTablet for tablet 7ad9722d782e420180336fe51de4f511 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3b7825bba42942e888ea189b445076a7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 00:51:29.472098 13017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ad9722d782e420180336fe51de4f511. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 00:51:29.479640 12681 tablet.cc:2369] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:51:29.485013 13159 tablet_bootstrap.cc:492] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2: Bootstrap starting.
I20260810 00:51:29.488374 13159 tablet_bootstrap.cc:654] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:29.490622 13157 tablet_bootstrap.cc:492] T 7ad9722d782e420180336fe51de4f511 P b401ff211e244d0da16bf35a1e68e931: Bootstrap starting.
I20260810 00:51:29.490819 13158 tablet_bootstrap.cc:492] T 7ad9722d782e420180336fe51de4f511 P e3c5dce1cb384204acc347d4b3e2a29d: Bootstrap starting.
I20260810 00:51:29.494042 13157 tablet_bootstrap.cc:654] T 7ad9722d782e420180336fe51de4f511 P b401ff211e244d0da16bf35a1e68e931: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:29.496002 13158 tablet_bootstrap.cc:654] T 7ad9722d782e420180336fe51de4f511 P e3c5dce1cb384204acc347d4b3e2a29d: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:29.505903 13157 tablet_bootstrap.cc:492] T 7ad9722d782e420180336fe51de4f511 P b401ff211e244d0da16bf35a1e68e931: No bootstrap required, opened a new log
I20260810 00:51:29.506191 13157 ts_tablet_manager.cc:1397] T 7ad9722d782e420180336fe51de4f511 P b401ff211e244d0da16bf35a1e68e931: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.000s
I20260810 00:51:29.507596 13157 raft_consensus.cc:348] T 7ad9722d782e420180336fe51de4f511 P b401ff211e244d0da16bf35a1e68e931 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.508138 13157 raft_consensus.cc:374] T 7ad9722d782e420180336fe51de4f511 P b401ff211e244d0da16bf35a1e68e931 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:29.508276 13157 raft_consensus.cc:729] T 7ad9722d782e420180336fe51de4f511 P b401ff211e244d0da16bf35a1e68e931 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b401ff211e244d0da16bf35a1e68e931, State: Initialized, Role: FOLLOWER
I20260810 00:51:29.510084 13157 consensus_queue.cc:260] T 7ad9722d782e420180336fe51de4f511 P b401ff211e244d0da16bf35a1e68e931 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.510852 13158 tablet_bootstrap.cc:492] T 7ad9722d782e420180336fe51de4f511 P e3c5dce1cb384204acc347d4b3e2a29d: No bootstrap required, opened a new log
I20260810 00:51:29.511113 13158 ts_tablet_manager.cc:1397] T 7ad9722d782e420180336fe51de4f511 P e3c5dce1cb384204acc347d4b3e2a29d: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.001s
I20260810 00:51:29.512143 13165 tablet_replica.cc:439] T 7ad9722d782e420180336fe51de4f511 P b401ff211e244d0da16bf35a1e68e931: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } } }
I20260810 00:51:29.512748 13165 tablet_replica.cc:442] T 7ad9722d782e420180336fe51de4f511 P b401ff211e244d0da16bf35a1e68e931: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:51:29.512741 13158 raft_consensus.cc:348] T 7ad9722d782e420180336fe51de4f511 P e3c5dce1cb384204acc347d4b3e2a29d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.513226 13158 raft_consensus.cc:374] T 7ad9722d782e420180336fe51de4f511 P e3c5dce1cb384204acc347d4b3e2a29d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:29.513377 13158 raft_consensus.cc:729] T 7ad9722d782e420180336fe51de4f511 P e3c5dce1cb384204acc347d4b3e2a29d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c5dce1cb384204acc347d4b3e2a29d, State: Initialized, Role: FOLLOWER
I20260810 00:51:29.513902 13158 consensus_queue.cc:260] T 7ad9722d782e420180336fe51de4f511 P e3c5dce1cb384204acc347d4b3e2a29d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.514715 13166 tablet_replica.cc:439] T 7ad9722d782e420180336fe51de4f511 P e3c5dce1cb384204acc347d4b3e2a29d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } } }
I20260810 00:51:29.515117 13158 ts_tablet_manager.cc:1428] T 7ad9722d782e420180336fe51de4f511 P e3c5dce1cb384204acc347d4b3e2a29d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 00:51:29.515340 13166 tablet_replica.cc:442] T 7ad9722d782e420180336fe51de4f511 P e3c5dce1cb384204acc347d4b3e2a29d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:51:29.516389 13157 ts_tablet_manager.cc:1428] T 7ad9722d782e420180336fe51de4f511 P b401ff211e244d0da16bf35a1e68e931: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260810 00:51:29.520506 13159 tablet_bootstrap.cc:492] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2: No bootstrap required, opened a new log
I20260810 00:51:29.520921 13159 ts_tablet_manager.cc:1397] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2: Time spent bootstrapping tablet: real 0.036s user 0.010s sys 0.000s
I20260810 00:51:29.522931 13159 raft_consensus.cc:348] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.523656 13159 raft_consensus.cc:374] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:29.523940 13159 raft_consensus.cc:729] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5addb1ac9954a768ea5015e913f13e2, State: Initialized, Role: FOLLOWER
I20260810 00:51:29.524811 13159 consensus_queue.cc:260] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.526436 13159 ts_tablet_manager.cc:1428] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260810 00:51:29.526026 13163 tablet_replica.cc:439] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } } }
I20260810 00:51:29.527029 13163 tablet_replica.cc:442] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:51:29.722364 13163 raft_consensus.cc:482] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:51:29.722672 13163 raft_consensus.cc:504] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.723721 13163 leader_election.cc:288] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3c5dce1cb384204acc347d4b3e2a29d (127.10.97.65:39961), b401ff211e244d0da16bf35a1e68e931 (127.10.97.67:37809)
I20260810 00:51:29.724483 12607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad9722d782e420180336fe51de4f511" candidate_uuid: "f5addb1ac9954a768ea5015e913f13e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" is_pre_election: true
I20260810 00:51:29.724750 13050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad9722d782e420180336fe51de4f511" candidate_uuid: "f5addb1ac9954a768ea5015e913f13e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b401ff211e244d0da16bf35a1e68e931" is_pre_election: true
I20260810 00:51:29.724968 12607 raft_consensus.cc:2454] T 7ad9722d782e420180336fe51de4f511 P e3c5dce1cb384204acc347d4b3e2a29d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5addb1ac9954a768ea5015e913f13e2 in term 0.
I20260810 00:51:29.725242 13050 raft_consensus.cc:2454] T 7ad9722d782e420180336fe51de4f511 P b401ff211e244d0da16bf35a1e68e931 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5addb1ac9954a768ea5015e913f13e2 in term 0.
I20260810 00:51:29.725737 12708 leader_election.cc:302] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e3c5dce1cb384204acc347d4b3e2a29d, f5addb1ac9954a768ea5015e913f13e2; no voters:
I20260810 00:51:29.726195 13163 raft_consensus.cc:2781] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:51:29.726374 13163 raft_consensus.cc:482] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:51:29.726583 13163 raft_consensus.cc:3037] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:29.730218 13163 raft_consensus.cc:504] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.731587 13163 leader_election.cc:288] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 [CANDIDATE]: Term 1 election: Requested vote from peers e3c5dce1cb384204acc347d4b3e2a29d (127.10.97.65:39961), b401ff211e244d0da16bf35a1e68e931 (127.10.97.67:37809)
I20260810 00:51:29.732266 12607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad9722d782e420180336fe51de4f511" candidate_uuid: "f5addb1ac9954a768ea5015e913f13e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c5dce1cb384204acc347d4b3e2a29d"
I20260810 00:51:29.732610 12607 raft_consensus.cc:3037] T 7ad9722d782e420180336fe51de4f511 P e3c5dce1cb384204acc347d4b3e2a29d [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:29.732504 13050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad9722d782e420180336fe51de4f511" candidate_uuid: "f5addb1ac9954a768ea5015e913f13e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b401ff211e244d0da16bf35a1e68e931"
I20260810 00:51:29.732975 13050 raft_consensus.cc:3037] T 7ad9722d782e420180336fe51de4f511 P b401ff211e244d0da16bf35a1e68e931 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:29.735719 12607 raft_consensus.cc:2454] T 7ad9722d782e420180336fe51de4f511 P e3c5dce1cb384204acc347d4b3e2a29d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5addb1ac9954a768ea5015e913f13e2 in term 1.
I20260810 00:51:29.736362 12708 leader_election.cc:302] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e3c5dce1cb384204acc347d4b3e2a29d, f5addb1ac9954a768ea5015e913f13e2; no voters:
I20260810 00:51:29.736780 13163 raft_consensus.cc:2781] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:29.737416 13163 raft_consensus.cc:686] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 [term 1 LEADER]: Becoming Leader. State: Replica: f5addb1ac9954a768ea5015e913f13e2, State: Running, Role: LEADER
I20260810 00:51:29.737924 13163 consensus_queue.cc:237] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } }
I20260810 00:51:29.738782 13050 raft_consensus.cc:2454] T 7ad9722d782e420180336fe51de4f511 P b401ff211e244d0da16bf35a1e68e931 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5addb1ac9954a768ea5015e913f13e2 in term 1.
I20260810 00:51:29.742059 13223 tablet_replica.cc:439] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2: TxnStatusTablet state changed. Reason: New leader f5addb1ac9954a768ea5015e913f13e2. Latest consensus state: current_term: 1 leader_uuid: "f5addb1ac9954a768ea5015e913f13e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } } }
I20260810 00:51:29.742724 13223 tablet_replica.cc:442] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:51:29.742414 12367 catalog_manager.cc:5495] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5addb1ac9954a768ea5015e913f13e2 (127.10.97.66). New cstate: current_term: 1 leader_uuid: "f5addb1ac9954a768ea5015e913f13e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } health_report { overall_health: UNKNOWN } } }
I20260810 00:51:29.744082 13231 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 00:51:29.744575 13231 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 00:51:29.794298 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.785913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36516 (local address 127.10.97.66:35685)
0810 00:51:29.786211 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:29.786233 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:29.786452 (+ 219us) server_negotiation.cc:407] Connection header received
0810 00:51:29.786874 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:29.786879 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:29.786993 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:29.787143 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:51:29.787906 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.789147 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.791285 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.792092 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.792312 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.793041 (+ 729us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:51:29.793244 (+ 203us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:51:29.793389 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:29.793601 (+ 212us) server_negotiation.cc:299] Negotiation successful
0810 00:51:29.793740 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 00:51:29.794474 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.785942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:35685 (local address 127.0.0.1:36516)
0810 00:51:29.786313 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:29.786368 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:29.786590 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:29.787199 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:29.787205 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:29.787240 (+ 35us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:51:29.787645 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.787665 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.789367 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.789376 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.791073 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.791101 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.792373 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.792381 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.792626 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.792735 (+ 109us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:51:29.793389 (+ 654us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:51:29.793433 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 00:51:29.793568 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 00:51:29.793877 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 00:51:29.801273 13050 raft_consensus.cc:1261] T 7ad9722d782e420180336fe51de4f511 P b401ff211e244d0da16bf35a1e68e931 [term 1 FOLLOWER]: Refusing update from remote peer f5addb1ac9954a768ea5015e913f13e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:51:29.801717 12607 raft_consensus.cc:1261] T 7ad9722d782e420180336fe51de4f511 P e3c5dce1cb384204acc347d4b3e2a29d [term 1 FOLLOWER]: Refusing update from remote peer f5addb1ac9954a768ea5015e913f13e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:51:29.802172 13223 consensus_queue.cc:1035] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:51:29.802781 13163 consensus_queue.cc:1035] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:51:29.808260 13165 tablet_replica.cc:439] T 7ad9722d782e420180336fe51de4f511 P b401ff211e244d0da16bf35a1e68e931: TxnStatusTablet state changed. Reason: New leader f5addb1ac9954a768ea5015e913f13e2. Latest consensus state: current_term: 1 leader_uuid: "f5addb1ac9954a768ea5015e913f13e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } } }
I20260810 00:51:29.808737 13165 tablet_replica.cc:442] T 7ad9722d782e420180336fe51de4f511 P b401ff211e244d0da16bf35a1e68e931: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:51:29.820093 13165 tablet_replica.cc:439] T 7ad9722d782e420180336fe51de4f511 P b401ff211e244d0da16bf35a1e68e931: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5addb1ac9954a768ea5015e913f13e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } } }
I20260810 00:51:29.820585 13165 tablet_replica.cc:442] T 7ad9722d782e420180336fe51de4f511 P b401ff211e244d0da16bf35a1e68e931: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:51:29.826924 13163 tablet_replica.cc:439] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f5addb1ac9954a768ea5015e913f13e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } } }
I20260810 00:51:29.827488 13163 tablet_replica.cc:442] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:51:29.836338 13204 consensus_queue.cc:1035] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:51:29.837113 13166 tablet_replica.cc:439] T 7ad9722d782e420180336fe51de4f511 P e3c5dce1cb384204acc347d4b3e2a29d: TxnStatusTablet state changed. Reason: New leader f5addb1ac9954a768ea5015e913f13e2. Latest consensus state: current_term: 1 leader_uuid: "f5addb1ac9954a768ea5015e913f13e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } } }
I20260810 00:51:29.837641 13166 tablet_replica.cc:442] T 7ad9722d782e420180336fe51de4f511 P e3c5dce1cb384204acc347d4b3e2a29d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:51:29.840476 13163 tablet_replica.cc:439] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f5addb1ac9954a768ea5015e913f13e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } } }
I20260810 00:51:29.840922 13163 tablet_replica.cc:442] T 7ad9722d782e420180336fe51de4f511 P f5addb1ac9954a768ea5015e913f13e2: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:51:29.849495 13166 tablet_replica.cc:439] T 7ad9722d782e420180336fe51de4f511 P e3c5dce1cb384204acc347d4b3e2a29d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5addb1ac9954a768ea5015e913f13e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c5dce1cb384204acc347d4b3e2a29d" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39961 } } peers { permanent_uuid: "f5addb1ac9954a768ea5015e913f13e2" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35685 } } peers { permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 } } }
I20260810 00:51:29.852936 13166 tablet_replica.cc:442] T 7ad9722d782e420180336fe51de4f511 P e3c5dce1cb384204acc347d4b3e2a29d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:51:29.865037 13204 consensus_queue.cc:1035] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b401ff211e244d0da16bf35a1e68e931" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:51:29.877084 13249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.857337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43994 (local address 127.10.97.65:39961)
0810 00:51:29.859567 (+ 2230us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:29.859577 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:29.862747 (+ 3170us) server_negotiation.cc:407] Connection header received
0810 00:51:29.863337 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:29.863356 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:29.863539 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:29.863787 (+ 248us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:51:29.865927 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.867843 (+ 1916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.871595 (+ 3752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.872931 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.874076 (+ 1145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.874362 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:51:29.875689 (+ 1327us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:51:29.875968 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:29.876154 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 00:51:29.876323 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1967,"thread_start_us":1732,"threads_started":1}
I20260810 00:51:29.904122 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.894856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:32889 (local address 127.10.97.66:35685)
0810 00:51:29.895175 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:29.895192 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:29.895979 (+ 787us) server_negotiation.cc:407] Connection header received
0810 00:51:29.896232 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:29.896259 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:29.896435 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:29.896614 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:51:29.897306 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.898641 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.901024 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.901849 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.902057 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.902735 (+ 678us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:51:29.902942 (+ 207us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:51:29.903096 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:29.903344 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 00:51:29.903518 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260810 00:51:29.904287 13255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:29.894809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:35685 (local address 127.10.97.65:32889)
0810 00:51:29.895796 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:29.895871 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:29.896046 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:29.896640 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:29.896645 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:29.896655 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:51:29.897049 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.897067 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.898881 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.898898 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.900833 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:29.900856 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:29.902108 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:29.902115 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:29.902355 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:29.902461 (+ 106us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:51:29.903083 (+ 622us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:51:29.903115 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 00:51:29.903267 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 00:51:29.903544 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":455,"threads_started":1}
I20260810 00:51:30.064670 13249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:30.027307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:59359 (local address 127.10.97.65:39961)
0810 00:51:30.027611 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:30.027622 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:30.042168 (+ 14546us) server_negotiation.cc:407] Connection header received
0810 00:51:30.044375 (+ 2207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:30.044383 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:30.044553 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:30.044682 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:51:30.046183 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:30.048151 (+ 1968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:30.059905 (+ 11754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:30.061166 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:30.061366 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:30.062844 (+ 1478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:51:30.063138 (+ 294us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:51:30.063310 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:30.063751 (+ 441us) server_negotiation.cc:299] Negotiation successful
0810 00:51:30.063970 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 00:51:30.070325 13262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:30.031704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:39961 (local address 127.10.97.66:59359)
0810 00:51:30.041995 (+ 10291us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:30.042036 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:30.044043 (+ 2007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:30.045180 (+ 1137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:30.045197 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:30.045251 (+ 54us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:51:30.045853 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:30.045887 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:30.052276 (+ 6389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:30.052284 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:30.055644 (+ 3360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:30.055684 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:30.061787 (+ 6103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:30.061813 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:30.062205 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:30.062402 (+ 197us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:51:30.063463 (+ 1061us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:51:30.063486 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 00:51:30.069219 (+ 5733us) client_negotiation.cc:240] Negotiation successful
0810 00:51:30.069533 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9802,"thread_start_us":11636,"threads_started":1}
I20260810 00:51:30.168583 12367 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:51:30.169461 12367 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:51:30.177040 12367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a768f4ee7274d0ab66c4556cf0478dd: Sending DeleteTablet for 3 replicas of tablet c0d9a68fda204278abc6a94accffde9b
I20260810 00:51:30.178359 12566 tablet_service.cc:1514] Processing DeleteTablet for tablet c0d9a68fda204278abc6a94accffde9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:30 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:43988
I20260810 00:51:30.181394 12778 tablet_service.cc:1514] Processing DeleteTablet for tablet c0d9a68fda204278abc6a94accffde9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:30 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:36506
I20260810 00:51:30.183457 13017 tablet_service.cc:1514] Processing DeleteTablet for tablet c0d9a68fda204278abc6a94accffde9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:30 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:51086
I20260810 00:51:30.194875 13271 tablet_replica.cc:330] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2: stopping tablet replica
I20260810 00:51:30.195288 13271 raft_consensus.cc:2229] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:51:30.195909 13271 raft_consensus.cc:2258] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:30.208317 13269 tablet_replica.cc:330] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d: stopping tablet replica
I20260810 00:51:30.209960 13271 ts_tablet_manager.cc:1905] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:30.210569 13269 raft_consensus.cc:2229] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:51:30.211769 13269 raft_consensus.cc:2258] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:30.213981 13269 ts_tablet_manager.cc:1905] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:30.216439 13272 tablet_replica.cc:330] T c0d9a68fda204278abc6a94accffde9b P b401ff211e244d0da16bf35a1e68e931: stopping tablet replica
I20260810 00:51:30.216843 13272 raft_consensus.cc:2229] T c0d9a68fda204278abc6a94accffde9b P b401ff211e244d0da16bf35a1e68e931 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:51:30.217345 13272 raft_consensus.cc:2258] T c0d9a68fda204278abc6a94accffde9b P b401ff211e244d0da16bf35a1e68e931 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:30.229045 13272 ts_tablet_manager.cc:1905] T c0d9a68fda204278abc6a94accffde9b P b401ff211e244d0da16bf35a1e68e931: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:30.238502 13269 ts_tablet_manager.cc:1918] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:51:30.239769 13271 ts_tablet_manager.cc:1918] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:51:30.240000 13271 log.cc:1196] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/c0d9a68fda204278abc6a94accffde9b
I20260810 00:51:30.240839 13271 ts_tablet_manager.cc:1939] T c0d9a68fda204278abc6a94accffde9b P f5addb1ac9954a768ea5015e913f13e2: Deleting consensus metadata
I20260810 00:51:30.241616 13272 ts_tablet_manager.cc:1918] T c0d9a68fda204278abc6a94accffde9b P b401ff211e244d0da16bf35a1e68e931: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:51:30.241870 13272 log.cc:1196] T c0d9a68fda204278abc6a94accffde9b P b401ff211e244d0da16bf35a1e68e931: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/c0d9a68fda204278abc6a94accffde9b
I20260810 00:51:30.242749 13272 ts_tablet_manager.cc:1939] T c0d9a68fda204278abc6a94accffde9b P b401ff211e244d0da16bf35a1e68e931: Deleting consensus metadata
I20260810 00:51:30.242954 12351 catalog_manager.cc:4841] TS f5addb1ac9954a768ea5015e913f13e2 (127.10.97.66:35685): tablet c0d9a68fda204278abc6a94accffde9b (table test-table [id=9d83d80dfb6b4be48c90cd733b72b31f]) successfully deleted
I20260810 00:51:30.243382 13269 log.cc:1196] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/c0d9a68fda204278abc6a94accffde9b
I20260810 00:51:30.248739 13269 ts_tablet_manager.cc:1939] T c0d9a68fda204278abc6a94accffde9b P e3c5dce1cb384204acc347d4b3e2a29d: Deleting consensus metadata
I20260810 00:51:30.251106 12350 catalog_manager.cc:4841] TS e3c5dce1cb384204acc347d4b3e2a29d (127.10.97.65:39961): tablet c0d9a68fda204278abc6a94accffde9b (table test-table [id=9d83d80dfb6b4be48c90cd733b72b31f]) successfully deleted
Aug 10 00:51:30 dist-test-slave-jlzn krb5kdc[12308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323088, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:51:30.257017 12348 catalog_manager.cc:4841] TS b401ff211e244d0da16bf35a1e68e931 (127.10.97.67:37809): tablet c0d9a68fda204278abc6a94accffde9b (table test-table [id=9d83d80dfb6b4be48c90cd733b72b31f]) successfully deleted
I20260810 00:51:30.271281 13249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:30.214346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44000 (local address 127.10.97.65:39961)
0810 00:51:30.214695 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:30.214717 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:30.224723 (+ 10006us) server_negotiation.cc:407] Connection header received
0810 00:51:30.225077 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:30.225095 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:30.225266 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:30.225527 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:30.229566 (+ 4039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:30.231480 (+ 1914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:30.238850 (+ 7370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:30.240021 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:30.240278 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:30.255741 (+ 15463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:30.255768 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:30.255840 (+ 72us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:30.255897 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:30.263616 (+ 7719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:30.265305 (+ 1689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:30.265316 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:30.265319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:30.265611 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:30.266482 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:30.266505 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:30.266525 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:30.267391 (+ 866us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:30.268130 (+ 739us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:30.270168 (+ 2038us) server_negotiation.cc:299] Negotiation successful
0810 00:51:30.270567 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
W20260810 00:51:30.272876 12673 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44000
I20260810 00:51:30.338183 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:30.288885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39278 (local address 127.10.97.126:43905)
0810 00:51:30.308215 (+ 19330us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:30.308233 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:30.308266 (+ 33us) server_negotiation.cc:407] Connection header received
0810 00:51:30.308356 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:30.308373 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:30.308517 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:30.308718 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:30.312600 (+ 3882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:30.314556 (+ 1956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:30.317699 (+ 3143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:30.318857 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:30.319068 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:30.324764 (+ 5696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:30.324774 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:30.324789 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:30.324861 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:30.331022 (+ 6161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:30.332731 (+ 1709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:30.332738 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:30.332751 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:30.333010 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:30.333792 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:30.333801 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:30.333805 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:30.334471 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:30.334937 (+ 466us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:30.336939 (+ 2002us) server_negotiation.cc:299] Negotiation successful
0810 00:51:30.337341 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19091,"thread_start_us":22729,"threads_started":1}
W20260810 00:51:30.339053 12367 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39278
I20260810 00:51:30.344048 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 12476
I20260810 00:51:30.388064 13249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:30.349754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44004 (local address 127.10.97.65:39961)
0810 00:51:30.350042 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:30.350050 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:30.358546 (+ 8496us) server_negotiation.cc:407] Connection header received
0810 00:51:30.359252 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:30.359260 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:30.359407 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:30.359553 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:30.362608 (+ 3055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:30.364333 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:30.367459 (+ 3126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:30.368752 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:30.368931 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:30.375170 (+ 6239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:30.375190 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:30.375195 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:30.375278 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:30.382064 (+ 6786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:30.383588 (+ 1524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:30.383596 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:30.383599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:30.383911 (+ 312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:30.384718 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:30.384734 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:30.384738 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:30.385397 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:30.385797 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:30.386867 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0810 00:51:30.387351 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260810 00:51:30.390188 12673 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44004
W20260810 00:51:30.391183 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 12476: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:51:30.391541 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 12476
I20260810 00:51:30.445060 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 12686
Aug 10 00:51:30 dist-test-slave-jlzn krb5kdc[12308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323088, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.97.66@KRBTEST.COM
I20260810 00:51:30.496276 13282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:30.447736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36530 (local address 127.10.97.66:35685)
0810 00:51:30.460631 (+ 12895us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:30.460638 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:30.460681 (+ 43us) server_negotiation.cc:407] Connection header received
0810 00:51:30.460778 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:30.460793 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:30.461012 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:30.462033 (+ 1021us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:30.464124 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:30.466346 (+ 2222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:30.469799 (+ 3453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:30.471077 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:30.471383 (+ 306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:30.482505 (+ 11122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:30.482550 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:30.482599 (+ 49us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:30.482684 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:30.490236 (+ 7552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:30.491748 (+ 1512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:30.491772 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:30.491776 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:30.492182 (+ 406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:30.492976 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:30.492994 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:30.492998 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:30.493898 (+ 900us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:30.494414 (+ 516us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:30.495021 (+ 607us) server_negotiation.cc:299] Negotiation successful
0810 00:51:30.495418 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12665,"thread_start_us":12394,"threads_started":1}
W20260810 00:51:30.497602 12892 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36530
W20260810 00:51:30.498788 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 12686: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:51:30.499092 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 12686
I20260810 00:51:30.537324 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 12908
Aug 10 00:51:30 dist-test-slave-jlzn krb5kdc[12308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323088, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.97.67@KRBTEST.COM
I20260810 00:51:30.560818 13284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:30.538458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51102 (local address 127.10.97.67:37809)
0810 00:51:30.539295 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:30.539299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:30.539316 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:51:30.539417 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:30.539421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:30.539522 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:30.539699 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:30.541647 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:30.542830 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:30.544978 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:30.545681 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:30.545820 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:30.552380 (+ 6560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:30.552404 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:30.552436 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:30.552496 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:30.556833 (+ 4337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:30.558040 (+ 1207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:30.558049 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:30.558051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:30.558276 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:30.558706 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:30.558709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:30.558720 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:30.559283 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:30.559653 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:30.560046 (+ 393us) server_negotiation.cc:299] Negotiation successful
0810 00:51:30.560362 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":441,"threads_started":1}
W20260810 00:51:30.561834 13123 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51102
W20260810 00:51:30.562734 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 12908: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:51:30.562918 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 12908
I20260810 00:51:30.598927 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 12328
I20260810 00:51:30.600538 12433 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:30.757802 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 12328
2026-08-10T00:51:30Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5084 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:51:30 dist-test-slave-jlzn krb5kdc[13289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:51:30 dist-test-slave-jlzn krb5kdc[13289](info): set up 2 sockets
Aug 10 00:51:30 dist-test-slave-jlzn krb5kdc[13289](info): commencing operation
krb5kdc: starting...
W20260810 00:51:32.188346 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.360s user 0.007s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:51:32 dist-test-slave-jlzn krb5kdc[13289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:51:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:51:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:51:32.356101 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:38495
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:38495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:32.657341 13307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:32.657593 13307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:32.657718 13307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:32.664513 13307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:51:32.664603 13307 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 00:51:32.664669 13307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:32.664744 13307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:32.664805 13307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:51:32.664884 13307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:51:32.674091 13307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:38495
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:38495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:32.675098 13307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:32.676298 13307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:32.686851 13314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:32.686884 13318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:32.686918 13315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:32.687870 13307 server_base.cc:1034] running on GCE node
I20260810 00:51:32.688524 13307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:32.689664 13307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:32.690865 13307 hybrid_clock.cc:648] HybridClock initialized: now 1786323092690811 us; error 27 us; skew 500 ppm
I20260810 00:51:32.691159 13307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:32 dist-test-slave-jlzn krb5kdc[13289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323092, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:32.694078 13307 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:51:32.695376 13307 webserver.cc:460] Webserver started at http://127.10.97.126:33609/ using document root <none> and password file <none>
I20260810 00:51:32.696223 13307 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:32.696324 13307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:32.696657 13307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:32.699335 13307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "aa995c57e6374786acb73c8a5534e7cc"
format_stamp: "Formatted at 2026-08-10 00:51:32 on dist-test-slave-jlzn"
server_key: "3ce9b614ee9e69688e9f6febc159e604"
server_key_iv: "ac58949c0d4fd41cb88cb5f5dddee4fd"
server_key_version: "encryptionkey@0"
I20260810 00:51:32.700095 13307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "aa995c57e6374786acb73c8a5534e7cc"
format_stamp: "Formatted at 2026-08-10 00:51:32 on dist-test-slave-jlzn"
server_key: "3ce9b614ee9e69688e9f6febc159e604"
server_key_iv: "ac58949c0d4fd41cb88cb5f5dddee4fd"
server_key_version: "encryptionkey@0"
I20260810 00:51:32.704739 13307 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:51:32.707955 13328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:32.709138 13307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:51:32.709290 13307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "aa995c57e6374786acb73c8a5534e7cc"
format_stamp: "Formatted at 2026-08-10 00:51:32 on dist-test-slave-jlzn"
server_key: "3ce9b614ee9e69688e9f6febc159e604"
server_key_iv: "ac58949c0d4fd41cb88cb5f5dddee4fd"
server_key_version: "encryptionkey@0"
I20260810 00:51:32.709474 13307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:32.727371 13307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:32.731149 13307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:32.731366 13307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:32.750402 13307 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:38495
I20260810 00:51:32.750419 13418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:38495 every 8 connection(s)
I20260810 00:51:32.751590 13307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:51:32.754925 13419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:32.758442 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 13307
I20260810 00:51:32.758642 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:51:32.759459 10629 external_mini_cluster.cc:1428] Setting key 16c39c3ec4b44342a4b545c1eb73cc2e
I20260810 00:51:32.767613 13419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc: Bootstrap starting.
I20260810 00:51:32.771159 13419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:32.772431 13419 log.cc:824] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc: Log is configured to *not* fsync() on all Append() calls
Aug 10 00:51:32 dist-test-slave-jlzn krb5kdc[13289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:32.775243 13419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc: No bootstrap required, opened a new log
I20260810 00:51:32.781107 13424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:32.761456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37482 (local address 127.10.97.126:38495)
0810 00:51:32.762145 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:32.762156 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:32.762195 (+ 39us) server_negotiation.cc:407] Connection header received
0810 00:51:32.763724 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:32.763745 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:32.764130 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:32.764431 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:32.766105 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:32.767312 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:32.768989 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:32.769643 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:32.769763 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:32.774442 (+ 4679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:32.774450 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:32.774462 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:32.774497 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:32.777376 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:32.778119 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:32.778128 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:32.778133 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:32.778303 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:32.778694 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:32.778697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:32.778699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:32.779179 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:32.779386 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:32.779866 (+ 480us) server_negotiation.cc:299] Negotiation successful
0810 00:51:32.780187 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":310,"threads_started":1}
I20260810 00:51:32.783918 13419 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa995c57e6374786acb73c8a5534e7cc" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 38495 } }
I20260810 00:51:32.784245 13419 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:32.784332 13419 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa995c57e6374786acb73c8a5534e7cc, State: Initialized, Role: FOLLOWER
I20260810 00:51:32.784857 13419 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa995c57e6374786acb73c8a5534e7cc" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 38495 } }
I20260810 00:51:32.785087 13419 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:51:32.785214 13419 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:51:32.785365 13419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:32.787350 13419 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa995c57e6374786acb73c8a5534e7cc" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 38495 } }
I20260810 00:51:32.787904 13419 leader_election.cc:302] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa995c57e6374786acb73c8a5534e7cc; no voters:
I20260810 00:51:32.788424 13419 leader_election.cc:288] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:51:32.788524 13426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:32.789193 13426 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc [term 1 LEADER]: Becoming Leader. State: Replica: aa995c57e6374786acb73c8a5534e7cc, State: Running, Role: LEADER
I20260810 00:51:32.789698 13426 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa995c57e6374786acb73c8a5534e7cc" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 38495 } }
I20260810 00:51:32.790206 13419 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:51:32.792436 13428 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa995c57e6374786acb73c8a5534e7cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa995c57e6374786acb73c8a5534e7cc" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 38495 } } }
I20260810 00:51:32.792822 13428 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc [sys.catalog]: This master's current role is: LEADER
I20260810 00:51:32.794107 13442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:51:32.794050 13429 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa995c57e6374786acb73c8a5534e7cc. Latest consensus state: current_term: 1 leader_uuid: "aa995c57e6374786acb73c8a5534e7cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa995c57e6374786acb73c8a5534e7cc" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 38495 } } }
I20260810 00:51:32.794334 13429 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc [sys.catalog]: This master's current role is: LEADER
I20260810 00:51:32.798794 13442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:51:32.807140 13442 catalog_manager.cc:1347] Generated new cluster ID: 9f8e20bae95a4546a0ee246874af8058
I20260810 00:51:32.807233 13442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:51:32.832283 13442 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:51:32.833559 13442 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:51:32.843441 13442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc: Generated new TSK 0
I20260810 00:51:32.844231 13442 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
I20260810 00:51:32.888833 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:38495
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:33.194228 13463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:33.194473 13463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:33.194605 13463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:33.201718 13463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:33.201812 13463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:33.201929 13463 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 00:51:33.201989 13463 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 00:51:33.202088 13463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:51:33.211300 13463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.97.126:38495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:33.212484 13463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:33.213757 13463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:33.225461 13472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:33.225522 13478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:33.225548 13473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:33.226841 13463 server_base.cc:1034] running on GCE node
I20260810 00:51:33.227347 13463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:33.228192 13463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:33.229383 13463 hybrid_clock.cc:648] HybridClock initialized: now 1786323093229337 us; error 38 us; skew 500 ppm
I20260810 00:51:33.229664 13463 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:33 dist-test-slave-jlzn krb5kdc[13289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323093, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:33.232590 13463 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:51:33.233824 13463 webserver.cc:460] Webserver started at http://127.10.97.65:42929/ using document root <none> and password file <none>
I20260810 00:51:33.234436 13463 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:33.234576 13463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:33.234877 13463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:33.237489 13463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "909baa319b8e46f7bf50d9a57fe1da8f"
format_stamp: "Formatted at 2026-08-10 00:51:33 on dist-test-slave-jlzn"
server_key: "af72a0a4c9e1d1d9b9340355212c03a4"
server_key_iv: "564257f3c24f4e69d30a7db9615e0845"
server_key_version: "encryptionkey@0"
I20260810 00:51:33.238162 13463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "909baa319b8e46f7bf50d9a57fe1da8f"
format_stamp: "Formatted at 2026-08-10 00:51:33 on dist-test-slave-jlzn"
server_key: "af72a0a4c9e1d1d9b9340355212c03a4"
server_key_iv: "564257f3c24f4e69d30a7db9615e0845"
server_key_version: "encryptionkey@0"
I20260810 00:51:33.242858 13463 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 00:51:33.245929 13487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:33.247063 13463 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 00:51:33.247203 13463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "909baa319b8e46f7bf50d9a57fe1da8f"
format_stamp: "Formatted at 2026-08-10 00:51:33 on dist-test-slave-jlzn"
server_key: "af72a0a4c9e1d1d9b9340355212c03a4"
server_key_iv: "564257f3c24f4e69d30a7db9615e0845"
server_key_version: "encryptionkey@0"
I20260810 00:51:33.247366 13463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:33.264580 13463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:33.268318 13463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:33.268560 13463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:33.270929 13463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:33.271040 13463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:33.271193 13463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:33.271288 13463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 00:51:33 dist-test-slave-jlzn krb5kdc[13289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323093, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:33.292476 13424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:33.271271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:53177 (local address 127.10.97.126:38495)
0810 00:51:33.271456 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:33.271462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:33.272771 (+ 1309us) server_negotiation.cc:407] Connection header received
0810 00:51:33.275420 (+ 2649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:33.275425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:33.275508 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:33.275619 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:33.277835 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:33.278713 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:33.281361 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:33.281775 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:33.281869 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:33.286002 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:33.286010 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:33.286015 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:33.286058 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:33.289398 (+ 3340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:33.290085 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:33.290091 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:33.290095 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:33.290259 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:33.290664 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:33.290669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:33.290672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:33.291072 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:33.291231 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:33.291947 (+ 716us) server_negotiation.cc:299] Negotiation successful
0810 00:51:33.292157 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 00:51:33.293362 13499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:33.271748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:38495 (local address 127.10.97.65:53177)
0810 00:51:33.272610 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:33.272657 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:33.275151 (+ 2494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:33.275902 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:33.275914 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:33.276693 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:33.277638 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:33.277666 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:33.278884 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:33.278888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:33.281220 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:33.281237 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:33.281981 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:33.281988 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:33.282267 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:33.282935 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:33.282963 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:33.285764 (+ 2801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:33.289540 (+ 3776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:33.289549 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:33.289566 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:33.289957 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:33.290384 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:33.290387 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:33.290390 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:33.290568 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:33.291203 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:33.291214 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:33.291573 (+ 359us) client_negotiation.cc:769] Sending connection context
0810 00:51:33.291826 (+ 253us) client_negotiation.cc:240] Negotiation successful
0810 00:51:33.292298 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":322,"threads_started":1}
I20260810 00:51:33.314400 13569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:33.298820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:38495 (local address 127.10.97.65:40189)
0810 00:51:33.299379 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:33.299397 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:33.299491 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:33.299749 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:33.299752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:33.300141 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:33.300463 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:33.300474 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:33.301794 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:33.301797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:33.302724 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:33.302735 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:33.303639 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:33.303645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:33.303889 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:33.307842 (+ 3953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:33.307864 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:33.308990 (+ 1126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:33.311475 (+ 2485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:33.311481 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:33.311485 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:33.312033 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:33.312478 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:33.312482 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:33.312485 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:33.312659 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:33.313249 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:33.313258 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:33.313467 (+ 209us) client_negotiation.cc:769] Sending connection context
0810 00:51:33.313653 (+ 186us) client_negotiation.cc:240] Negotiation successful
0810 00:51:33.313963 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":306,"threads_started":1}
I20260810 00:51:33.315150 13424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:33.298751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:40189 (local address 127.10.97.126:38495)
0810 00:51:33.298883 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:33.298886 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:33.299479 (+ 593us) server_negotiation.cc:407] Connection header received
0810 00:51:33.299569 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:33.299574 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:33.299644 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:33.299740 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:33.300589 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:33.301680 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:33.302836 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:33.303435 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:33.303554 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:33.309170 (+ 5616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:33.309176 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:33.309180 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:33.309208 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:33.311355 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:33.312155 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:33.312160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:33.312163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:33.312311 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:33.312814 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:33.312819 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:33.312822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:33.313133 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:33.313259 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:33.314706 (+ 1447us) server_negotiation.cc:299] Negotiation successful
0810 00:51:33.314916 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 00:51:33.322683 13463 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:42495
I20260810 00:51:33.322726 13657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:42495 every 8 connection(s)
I20260810 00:51:33.323784 13463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:51:33.325707 13658 heartbeater.cc:347] Connected to a master server at 127.10.97.126:38495
I20260810 00:51:33.325909 13658 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:33.326462 13658 heartbeater.cc:511] Master 127.10.97.126:38495 requested a full tablet report, sending...
I20260810 00:51:33.328204 13355 ts_manager.cc:194] Registered new tserver with Master: 909baa319b8e46f7bf50d9a57fe1da8f (127.10.97.65:42495)
I20260810 00:51:33.329548 13355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:53177
I20260810 00:51:33.333557 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 13463
I20260810 00:51:33.333717 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:51:33.334290 10629 external_mini_cluster.cc:1428] Setting key 85588a8ee3cbfbf3931e297f0b06298e
WARNING: no policy specified for kudu/127.10.97.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
I20260810 00:51:33.373970 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:38495
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:33.692312 13664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:33.692566 13664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:33.692672 13664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:33.699729 13664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:33.699854 13664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:33.699950 13664 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 00:51:33.700004 13664 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 00:51:33.700070 13664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:51:33.709187 13664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.97.126:38495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:33.710269 13664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:33.711480 13664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:33.723696 13678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:33.723730 13673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:33.723758 13670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:33.724720 13664 server_base.cc:1034] running on GCE node
I20260810 00:51:33.725289 13664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:33.726150 13664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:33.727348 13664 hybrid_clock.cc:648] HybridClock initialized: now 1786323093727298 us; error 37 us; skew 500 ppm
I20260810 00:51:33.727648 13664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:33 dist-test-slave-jlzn krb5kdc[13289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323093, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:33.730619 13664 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:51:33.731968 13664 webserver.cc:460] Webserver started at http://127.10.97.66:35805/ using document root <none> and password file <none>
I20260810 00:51:33.732600 13664 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:33.732721 13664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:33.733022 13664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:33.736019 13664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "5c065702f62545968a5cf6d05ff45a29"
format_stamp: "Formatted at 2026-08-10 00:51:33 on dist-test-slave-jlzn"
server_key: "4d55b0b5a52c31abfc6143c2b32b2627"
server_key_iv: "448ac0d79b8d1de584e8e3fc5ac34d49"
server_key_version: "encryptionkey@0"
I20260810 00:51:33.736752 13664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "5c065702f62545968a5cf6d05ff45a29"
format_stamp: "Formatted at 2026-08-10 00:51:33 on dist-test-slave-jlzn"
server_key: "4d55b0b5a52c31abfc6143c2b32b2627"
server_key_iv: "448ac0d79b8d1de584e8e3fc5ac34d49"
server_key_version: "encryptionkey@0"
I20260810 00:51:33.741552 13664 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:51:33.744781 13687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:33.746034 13664 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 00:51:33.746188 13664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "5c065702f62545968a5cf6d05ff45a29"
format_stamp: "Formatted at 2026-08-10 00:51:33 on dist-test-slave-jlzn"
server_key: "4d55b0b5a52c31abfc6143c2b32b2627"
server_key_iv: "448ac0d79b8d1de584e8e3fc5ac34d49"
server_key_version: "encryptionkey@0"
I20260810 00:51:33.746367 13664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:33.775182 13664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:33.778764 13664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:33.779033 13664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:33.781598 13664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:33.781716 13664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:33.781834 13664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:33.781916 13664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 00:51:33 dist-test-slave-jlzn krb5kdc[13289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323093, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:33.805639 13424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:33.781894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:56629 (local address 127.10.97.126:38495)
0810 00:51:33.782078 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:33.782085 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:33.783425 (+ 1340us) server_negotiation.cc:407] Connection header received
0810 00:51:33.785838 (+ 2413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:33.785844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:33.785934 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:33.786070 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:33.788231 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:33.789241 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:33.791740 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:33.792216 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:33.792351 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:33.796934 (+ 4583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:33.796941 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:33.796946 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:33.796986 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:33.800263 (+ 3277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:33.801243 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:33.801249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:33.801253 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:33.801400 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:33.801980 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:33.801986 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:33.802003 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:33.802387 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:33.802541 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:33.805012 (+ 2471us) server_negotiation.cc:299] Negotiation successful
0810 00:51:33.805289 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 00:51:33.805649 13698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:33.782362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:38495 (local address 127.10.97.66:56629)
0810 00:51:33.783259 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:33.783303 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:33.785545 (+ 2242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:33.786353 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:33.786363 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:33.787073 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:33.787970 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:33.788000 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:33.789413 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:33.789419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:33.791601 (+ 2182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:33.791614 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:33.792458 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:33.792470 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:33.792791 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:33.793579 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:33.793615 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:33.796693 (+ 3078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:33.800404 (+ 3711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:33.800412 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:33.800426 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:33.801116 (+ 690us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:33.801517 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:33.801523 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:33.801527 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:33.801805 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:33.802625 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:33.802633 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:33.803164 (+ 531us) client_negotiation.cc:769] Sending connection context
0810 00:51:33.803511 (+ 347us) client_negotiation.cc:240] Negotiation successful
0810 00:51:33.804156 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":402,"threads_started":1}
I20260810 00:51:33.824365 13424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:33.810804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:58367 (local address 127.10.97.126:38495)
0810 00:51:33.811000 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:33.811005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:33.812016 (+ 1011us) server_negotiation.cc:407] Connection header received
0810 00:51:33.812182 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:33.812187 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:33.812263 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:33.812370 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:33.813554 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:33.814770 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:33.816112 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:33.816667 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:33.816782 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:33.819358 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:33.819364 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:33.819368 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:33.819397 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:33.821894 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:33.822500 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:33.822504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:33.822507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:33.822624 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:33.822956 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:33.822960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:33.822964 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:33.823265 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:33.823394 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:33.823785 (+ 391us) server_negotiation.cc:299] Negotiation successful
0810 00:51:33.824013 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 00:51:33.824515 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:33.810808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:38495 (local address 127.10.97.66:58367)
0810 00:51:33.811883 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:33.811907 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:33.812047 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:33.812448 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:33.812453 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:33.813125 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:33.813426 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:33.813435 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:33.814907 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:33.814911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:33.815983 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:33.815995 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:33.817105 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:33.817117 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:33.817264 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:33.818082 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:33.818100 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:33.819191 (+ 1091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:33.822013 (+ 2822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:33.822018 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:33.822022 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:33.822392 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:33.822728 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:33.822734 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:33.822738 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:33.822858 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:33.823379 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:33.823384 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:33.823553 (+ 169us) client_negotiation.cc:769] Sending connection context
0810 00:51:33.823675 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 00:51:33.824016 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":889,"thread_start_us":708,"threads_started":1}
I20260810 00:51:33.834965 13664 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:35733
I20260810 00:51:33.835001 13865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:35733 every 8 connection(s)
I20260810 00:51:33.836162 13664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:51:33.837953 13866 heartbeater.cc:347] Connected to a master server at 127.10.97.126:38495
I20260810 00:51:33.838155 13866 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:33.838558 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 13664
I20260810 00:51:33.838724 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:51:33.838762 13866 heartbeater.cc:511] Master 127.10.97.126:38495 requested a full tablet report, sending...
I20260810 00:51:33.839159 10629 external_mini_cluster.cc:1428] Setting key 677f9a9f8f061b81d64b69e899010c0d
I20260810 00:51:33.840114 13355 ts_manager.cc:194] Registered new tserver with Master: 5c065702f62545968a5cf6d05ff45a29 (127.10.97.66:35733)
I20260810 00:51:33.841082 13355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:56629
WARNING: no policy specified for kudu/127.10.97.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
I20260810 00:51:33.879931 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:38495
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:34.184521 13872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:34.184767 13872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:34.184868 13872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:34.191764 13872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:34.191891 13872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:34.191982 13872 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 00:51:34.192034 13872 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 00:51:34.192101 13872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:51:34.201287 13872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.97.126:38495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:34.202353 13872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:34.203584 13872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:34.215545 13886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:34.215579 13881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:34.215605 13883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:34.216382 13872 server_base.cc:1034] running on GCE node
I20260810 00:51:34.216898 13872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:34.217687 13872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:34.218869 13872 hybrid_clock.cc:648] HybridClock initialized: now 1786323094218822 us; error 42 us; skew 500 ppm
I20260810 00:51:34.219115 13872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:34 dist-test-slave-jlzn krb5kdc[13289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323094, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:34.222040 13872 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:51:34.223333 13872 webserver.cc:460] Webserver started at http://127.10.97.67:46171/ using document root <none> and password file <none>
I20260810 00:51:34.223940 13872 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:34.224046 13872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:34.224300 13872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:34.226933 13872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "22d1fe55e79e41a9b2fcef79196dcdb9"
format_stamp: "Formatted at 2026-08-10 00:51:34 on dist-test-slave-jlzn"
server_key: "c8ab22a82b424b6c423cafdf8032ae48"
server_key_iv: "824d5012844be4bb8bba0c214fed6cb9"
server_key_version: "encryptionkey@0"
I20260810 00:51:34.227603 13872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "22d1fe55e79e41a9b2fcef79196dcdb9"
format_stamp: "Formatted at 2026-08-10 00:51:34 on dist-test-slave-jlzn"
server_key: "c8ab22a82b424b6c423cafdf8032ae48"
server_key_iv: "824d5012844be4bb8bba0c214fed6cb9"
server_key_version: "encryptionkey@0"
I20260810 00:51:34.232203 13872 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 00:51:34.235327 13897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:34.236599 13872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:51:34.236735 13872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "22d1fe55e79e41a9b2fcef79196dcdb9"
format_stamp: "Formatted at 2026-08-10 00:51:34 on dist-test-slave-jlzn"
server_key: "c8ab22a82b424b6c423cafdf8032ae48"
server_key_iv: "824d5012844be4bb8bba0c214fed6cb9"
server_key_version: "encryptionkey@0"
I20260810 00:51:34.236907 13872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:34.253326 13872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:34.257033 13872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:34.257284 13872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:34.259573 13872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:34.259668 13872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:34.259769 13872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:34.259877 13872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 10 00:51:34 dist-test-slave-jlzn krb5kdc[13289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323094, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:34.282028 13424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:34.260103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:43051 (local address 127.10.97.126:38495)
0810 00:51:34.260285 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:34.260291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:34.261595 (+ 1304us) server_negotiation.cc:407] Connection header received
0810 00:51:34.264293 (+ 2698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:34.264299 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:34.264380 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:34.264479 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:34.266559 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.267690 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.269812 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.270390 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.270509 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:34.275349 (+ 4840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:34.275356 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:34.275361 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:34.275401 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:34.278380 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:34.279310 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:34.279316 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:34.279319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:34.279495 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:34.280027 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:34.280032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:34.280035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:34.280440 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:34.280593 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:34.281604 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0810 00:51:34.281788 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 00:51:34.283635 13910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:34.260588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:38495 (local address 127.10.97.67:43051)
0810 00:51:34.261422 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:34.261467 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:34.263983 (+ 2516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:34.264861 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:34.264875 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:34.265626 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:34.266362 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:34.266390 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.267874 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.267879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:34.269676 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:34.269689 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.270814 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.270821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:34.271012 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:34.271647 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:34.271681 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:34.275105 (+ 3424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:34.278506 (+ 3401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:34.278515 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:34.278532 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:34.279147 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:34.279594 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:34.279599 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:34.279603 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:34.279874 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:34.280695 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:34.280704 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:34.281170 (+ 466us) client_negotiation.cc:769] Sending connection context
0810 00:51:34.281501 (+ 331us) client_negotiation.cc:240] Negotiation successful
0810 00:51:34.282033 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":326,"threads_started":1}
I20260810 00:51:34.296639 13872 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:39951
I20260810 00:51:34.296674 14091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:39951 every 8 connection(s)
I20260810 00:51:34.298189 13872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:51:34.300302 14092 heartbeater.cc:347] Connected to a master server at 127.10.97.126:38495
I20260810 00:51:34.300505 14092 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:34.301088 14092 heartbeater.cc:511] Master 127.10.97.126:38495 requested a full tablet report, sending...
I20260810 00:51:34.302440 13355 ts_manager.cc:194] Registered new tserver with Master: 22d1fe55e79e41a9b2fcef79196dcdb9 (127.10.97.67:39951)
I20260810 00:51:34.303454 13355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:43051
I20260810 00:51:34.303927 14061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:34.290113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:38495 (local address 127.10.97.67:32999)
0810 00:51:34.290803 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:34.290821 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:34.290937 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:34.291267 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:34.291272 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:34.291974 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:34.292431 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:34.292449 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.293593 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.293598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:34.295115 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:34.295127 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.296361 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.296369 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:34.296552 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:34.297114 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:34.297136 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:34.298066 (+ 930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:34.301467 (+ 3401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:34.301472 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:34.301476 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:34.301926 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:34.302362 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:34.302365 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:34.302368 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:34.302485 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:34.303048 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:34.303051 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:34.303193 (+ 142us) client_negotiation.cc:769] Sending connection context
0810 00:51:34.303298 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 00:51:34.303496 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":389,"threads_started":1}
I20260810 00:51:34.304004 13424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:34.290010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:32999 (local address 127.10.97.126:38495)
0810 00:51:34.290494 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:34.290500 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:34.290912 (+ 412us) server_negotiation.cc:407] Connection header received
0810 00:51:34.291030 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:34.291035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:34.291106 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:34.291201 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:34.292582 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.293429 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.295238 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.295928 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.296050 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:34.298217 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:34.298221 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:34.298224 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:34.298260 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:34.301347 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:34.302030 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:34.302036 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:34.302040 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:34.302239 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:34.302591 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:34.302595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:34.302599 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:34.302949 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:34.303109 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:34.303427 (+ 318us) server_negotiation.cc:299] Negotiation successful
0810 00:51:34.303622 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":364}
I20260810 00:51:34.304008 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 13872
I20260810 00:51:34.304193 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:51:34.304705 10629 external_mini_cluster.cc:1428] Setting key e281088201686146681685f5aa188462
I20260810 00:51:34.307456 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:51:34.331863 13658 heartbeater.cc:503] Master 127.10.97.126:38495 was elected leader, sending a full tablet report...
Aug 10 00:51:34 dist-test-slave-jlzn krb5kdc[13289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 00:51:34 dist-test-slave-jlzn krb5kdc[13289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:34.354342 13424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:34.341956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37492 (local address 127.10.97.126:38495)
0810 00:51:34.342159 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:34.342164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:34.342348 (+ 184us) server_negotiation.cc:407] Connection header received
0810 00:51:34.342478 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:34.342482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:34.342539 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:34.342614 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:34.343860 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.344613 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.345731 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.346146 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.346231 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:34.349503 (+ 3272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:34.349507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:34.349511 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:34.349540 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:34.352045 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:34.352605 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:34.352609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:34.352611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:34.352743 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:34.353084 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:34.353087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:34.353089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:34.353344 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:34.353559 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:34.353865 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 00:51:34.354081 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 00:51:34.357815 13355 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:51:34.364387 13355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:51:34.385893 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:34.375162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:39951 (local address 127.0.0.1:59228)
0810 00:51:34.376047 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:34.376094 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:34.376243 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:34.378127 (+ 1884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:34.378135 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:34.378164 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:34.378556 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:34.378568 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.380798 (+ 2230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.380804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:34.383065 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:34.383078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.384979 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.384986 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:34.385193 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:34.385272 (+ 79us) client_negotiation.cc:769] Sending connection context
0810 00:51:34.385371 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 00:51:34.385538 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":904,"threads_started":1}
I20260810 00:51:34.387225 14115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:34.375486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59228 (local address 127.10.97.67:39951)
0810 00:51:34.377522 (+ 2036us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:34.377531 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:34.377561 (+ 30us) server_negotiation.cc:407] Connection header received
0810 00:51:34.377671 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:34.377679 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:34.377907 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:34.378765 (+ 858us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:34.379050 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.380652 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.383269 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.384729 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.386433 (+ 1704us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:34.386681 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:34.386761 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 00:51:34.386920 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1889,"thread_start_us":1723,"threads_started":1}
I20260810 00:51:34.388959 14114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:34.375341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:35733 (local address 127.0.0.1:51508)
0810 00:51:34.377488 (+ 2147us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:34.377505 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:34.377693 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:34.378502 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:34.378507 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:34.378534 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:34.379967 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:34.379986 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.381752 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.381757 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:34.384899 (+ 3142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:34.384912 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.388241 (+ 3329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.388247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:34.388417 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:34.388449 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 00:51:34.388514 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 00:51:34.388644 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2009,"thread_start_us":1882,"threads_started":1}
I20260810 00:51:34.389686 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:34.375928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51508 (local address 127.10.97.66:35733)
0810 00:51:34.377024 (+ 1096us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:34.377032 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:34.377596 (+ 564us) server_negotiation.cc:407] Connection header received
0810 00:51:34.377882 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:34.377890 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:34.378090 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:34.378268 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:34.380180 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.381557 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.385071 (+ 3514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.387755 (+ 2684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.387905 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:34.388100 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:34.389259 (+ 1159us) server_negotiation.cc:299] Negotiation successful
0810 00:51:34.389420 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":946,"thread_start_us":997,"threads_started":1}
I20260810 00:51:34.391249 13986 tablet_service.cc:1467] Processing CreateTablet for tablet 8873359c84814724a0efed985064a0a3 (DEFAULT_TABLE table=test-table [id=5d81b554db2748c695190de8eddebcc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:34.392881 13986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8873359c84814724a0efed985064a0a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:34.393293 13766 tablet_service.cc:1467] Processing CreateTablet for tablet 8873359c84814724a0efed985064a0a3 (DEFAULT_TABLE table=test-table [id=5d81b554db2748c695190de8eddebcc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:34.394371 13766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8873359c84814724a0efed985064a0a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:34.394819 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:34.377571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42495 (local address 127.0.0.1:45054)
0810 00:51:34.386135 (+ 8564us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:34.386151 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:34.386241 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:34.388509 (+ 2268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:34.388513 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:34.388540 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:34.388689 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:34.388700 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.390734 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.390739 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:34.392432 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:34.392448 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.394134 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.394141 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:34.394315 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:34.394347 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 00:51:34.394441 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 00:51:34.394563 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8481,"thread_start_us":12512,"threads_started":1}
I20260810 00:51:34.394979 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:34.377816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45054 (local address 127.10.97.65:42495)
0810 00:51:34.387951 (+ 10135us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:34.387959 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:34.387987 (+ 28us) server_negotiation.cc:407] Connection header received
0810 00:51:34.388076 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:34.388083 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:34.388312 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:34.388865 (+ 553us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:34.389058 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.390553 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.392611 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.393953 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.394096 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:34.394290 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:34.394514 (+ 224us) server_negotiation.cc:299] Negotiation successful
0810 00:51:34.394664 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9956,"thread_start_us":12515,"threads_started":1}
I20260810 00:51:34.398330 13553 tablet_service.cc:1467] Processing CreateTablet for tablet 8873359c84814724a0efed985064a0a3 (DEFAULT_TABLE table=test-table [id=5d81b554db2748c695190de8eddebcc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:34.399547 13553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8873359c84814724a0efed985064a0a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:34.404767 14121 tablet_bootstrap.cc:492] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9: Bootstrap starting.
I20260810 00:51:34.407536 14121 tablet_bootstrap.cc:654] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:34.408493 14122 tablet_bootstrap.cc:492] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29: Bootstrap starting.
I20260810 00:51:34.408841 14121 log.cc:824] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:34.411666 14122 tablet_bootstrap.cc:654] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:34.412770 14125 tablet_bootstrap.cc:492] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f: Bootstrap starting.
I20260810 00:51:34.413132 14121 tablet_bootstrap.cc:492] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9: No bootstrap required, opened a new log
I20260810 00:51:34.413399 14121 ts_tablet_manager.cc:1397] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260810 00:51:34.413754 14122 log.cc:824] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:34.415850 14125 tablet_bootstrap.cc:654] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:34.416741 14125 log.cc:824] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:34.416777 14122 tablet_bootstrap.cc:492] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29: No bootstrap required, opened a new log
I20260810 00:51:34.417014 14122 ts_tablet_manager.cc:1397] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 00:51:34.418855 14125 tablet_bootstrap.cc:492] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f: No bootstrap required, opened a new log
I20260810 00:51:34.419068 14125 ts_tablet_manager.cc:1397] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20260810 00:51:34.423101 14121 raft_consensus.cc:348] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } }
I20260810 00:51:34.423537 14121 raft_consensus.cc:374] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:34.423646 14121 raft_consensus.cc:729] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22d1fe55e79e41a9b2fcef79196dcdb9, State: Initialized, Role: FOLLOWER
I20260810 00:51:34.424237 14121 consensus_queue.cc:260] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } }
I20260810 00:51:34.426048 14125 raft_consensus.cc:348] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } }
I20260810 00:51:34.426352 14125 raft_consensus.cc:374] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:34.426455 14125 raft_consensus.cc:729] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 909baa319b8e46f7bf50d9a57fe1da8f, State: Initialized, Role: FOLLOWER
I20260810 00:51:34.427129 14092 heartbeater.cc:503] Master 127.10.97.126:38495 was elected leader, sending a full tablet report...
I20260810 00:51:34.427089 14125 consensus_queue.cc:260] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } }
I20260810 00:51:34.427721 14121 ts_tablet_manager.cc:1428] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260810 00:51:34.428448 14122 raft_consensus.cc:348] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } }
I20260810 00:51:34.428848 14122 raft_consensus.cc:374] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:34.428942 14122 raft_consensus.cc:729] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c065702f62545968a5cf6d05ff45a29, State: Initialized, Role: FOLLOWER
I20260810 00:51:34.429407 14125 ts_tablet_manager.cc:1428] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f: Time spent starting tablet: real 0.010s user 0.008s sys 0.002s
I20260810 00:51:34.429563 14122 consensus_queue.cc:260] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } }
I20260810 00:51:34.430502 13866 heartbeater.cc:503] Master 127.10.97.126:38495 was elected leader, sending a full tablet report...
I20260810 00:51:34.431072 14122 ts_tablet_manager.cc:1428] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260810 00:51:34.510573 14128 raft_consensus.cc:482] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:51:34.510819 14128 raft_consensus.cc:504] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } }
I20260810 00:51:34.512406 14128 leader_election.cc:288] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c065702f62545968a5cf6d05ff45a29 (127.10.97.66:35733), 909baa319b8e46f7bf50d9a57fe1da8f (127.10.97.65:42495)
I20260810 00:51:34.517824 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:34.512728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:33549 (local address 127.10.97.66:35733)
0810 00:51:34.512889 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:34.512893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:34.512991 (+ 98us) server_negotiation.cc:407] Connection header received
0810 00:51:34.513235 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:34.513241 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:34.513313 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:34.513399 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:34.513934 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.514670 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.516256 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.517000 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.517151 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:34.517225 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:34.517466 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 00:51:34.517585 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 00:51:34.517815 13910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:34.512741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:35733 (local address 127.10.97.67:33549)
0810 00:51:34.512915 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:34.512934 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:34.513052 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:34.513569 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:34.513574 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:34.513599 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:34.513799 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:34.513808 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.514791 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.514797 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:34.516094 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:34.516106 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.517145 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.517151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:34.517318 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:34.517341 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 00:51:34.517419 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 00:51:34.517542 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":1}
I20260810 00:51:34.518729 14132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:34.513094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42495 (local address 127.10.97.67:47141)
0810 00:51:34.513774 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:34.513787 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:34.513908 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:34.514209 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:34.514213 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:34.514236 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:34.514536 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:34.514544 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.515513 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.515518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:34.516981 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:34.516994 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.518149 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.518156 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:34.518311 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:34.518344 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 00:51:34.518429 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 00:51:34.518510 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":338,"threads_started":1}
I20260810 00:51:34.518806 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:34.513177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:47141 (local address 127.10.97.65:42495)
0810 00:51:34.513360 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:34.513366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:34.513866 (+ 500us) server_negotiation.cc:407] Connection header received
0810 00:51:34.514029 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:34.514035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:34.514104 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:34.514182 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:34.514702 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.515386 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.517135 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.517848 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.518002 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:34.518062 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:34.518462 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 00:51:34.518583 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 00:51:34.518864 13803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8873359c84814724a0efed985064a0a3" candidate_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c065702f62545968a5cf6d05ff45a29" is_pre_election: true
I20260810 00:51:34.519392 13803 raft_consensus.cc:2454] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22d1fe55e79e41a9b2fcef79196dcdb9 in term 0.
I20260810 00:51:34.519654 13595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8873359c84814724a0efed985064a0a3" candidate_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" is_pre_election: true
I20260810 00:51:34.520004 13899 leader_election.cc:302] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22d1fe55e79e41a9b2fcef79196dcdb9, 5c065702f62545968a5cf6d05ff45a29; no voters:
I20260810 00:51:34.520139 13595 raft_consensus.cc:2454] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22d1fe55e79e41a9b2fcef79196dcdb9 in term 0.
I20260810 00:51:34.520346 14128 raft_consensus.cc:2781] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:51:34.520494 14128 raft_consensus.cc:482] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:51:34.520601 14128 raft_consensus.cc:3037] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:34.522717 14128 raft_consensus.cc:504] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } }
I20260810 00:51:34.523231 14128 leader_election.cc:288] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [CANDIDATE]: Term 1 election: Requested vote from peers 5c065702f62545968a5cf6d05ff45a29 (127.10.97.66:35733), 909baa319b8e46f7bf50d9a57fe1da8f (127.10.97.65:42495)
I20260810 00:51:34.523594 13803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8873359c84814724a0efed985064a0a3" candidate_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c065702f62545968a5cf6d05ff45a29"
I20260810 00:51:34.523705 13595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8873359c84814724a0efed985064a0a3" candidate_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "909baa319b8e46f7bf50d9a57fe1da8f"
I20260810 00:51:34.523823 13803 raft_consensus.cc:3037] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:34.523902 13595 raft_consensus.cc:3037] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:34.526837 13595 raft_consensus.cc:2454] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d1fe55e79e41a9b2fcef79196dcdb9 in term 1.
I20260810 00:51:34.526837 13803 raft_consensus.cc:2454] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d1fe55e79e41a9b2fcef79196dcdb9 in term 1.
I20260810 00:51:34.527174 13899 leader_election.cc:302] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22d1fe55e79e41a9b2fcef79196dcdb9, 909baa319b8e46f7bf50d9a57fe1da8f; no voters:
I20260810 00:51:34.527416 14128 raft_consensus.cc:2781] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:34.527844 14128 raft_consensus.cc:686] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 LEADER]: Becoming Leader. State: Replica: 22d1fe55e79e41a9b2fcef79196dcdb9, State: Running, Role: LEADER
I20260810 00:51:34.528268 14128 consensus_queue.cc:237] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } }
I20260810 00:51:34.532444 13353 catalog_manager.cc:5495] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22d1fe55e79e41a9b2fcef79196dcdb9 (127.10.97.67). New cstate: current_term: 1 leader_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } health_report { overall_health: UNKNOWN } } }
W20260810 00:51:34.550464 14093 tablet.cc:2369] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:51:34.576315 13660 tablet.cc:2369] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:51:34.587893 13868 tablet.cc:2369] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 00:51:34 dist-test-slave-jlzn krb5kdc[13289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323092, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:34.605156 14147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:34.589826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:38495 (local address 127.0.0.1:37498)
0810 00:51:34.590394 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:34.590408 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:34.590566 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:34.590854 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:34.590857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:34.591517 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:34.591835 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:34.591847 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.592788 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.592791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:34.593727 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:34.593738 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.594361 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.594366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:34.594491 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:34.598517 (+ 4026us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:34.598533 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:34.600570 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:34.603017 (+ 2447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:34.603027 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:34.603031 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:34.603427 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:34.603733 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:34.603737 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:34.603739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:34.603906 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:34.604349 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:34.604354 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:34.604496 (+ 142us) client_negotiation.cc:769] Sending connection context
0810 00:51:34.604623 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 00:51:34.604820 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":306,"threads_started":1}
I20260810 00:51:34.605194 13424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:34.589763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37498 (local address 127.10.97.126:38495)
0810 00:51:34.589948 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:34.589954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:34.590487 (+ 533us) server_negotiation.cc:407] Connection header received
0810 00:51:34.590689 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:34.590693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:34.590752 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:34.590821 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:34.591962 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.592673 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.593836 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.594228 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.594314 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:34.600743 (+ 6429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:34.600748 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:34.600751 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:34.600778 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:34.602906 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:34.603518 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:34.603521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:34.603524 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:34.603644 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:34.604002 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:34.604006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:34.604008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:34.604246 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:34.604350 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:34.604746 (+ 396us) server_negotiation.cc:299] Negotiation successful
0810 00:51:34.604918 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":53}
I20260810 00:51:34.609486 13353 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:37498:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 00:51:34.610394 13353 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:51:34.616286 13766 tablet_service.cc:1467] Processing CreateTablet for tablet 1d43ea16e7e04c4397a21fb875dc6c11 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=20c33900d26c4d7f926299ded665a5fe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 00:51:34.616362 13553 tablet_service.cc:1467] Processing CreateTablet for tablet 1d43ea16e7e04c4397a21fb875dc6c11 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=20c33900d26c4d7f926299ded665a5fe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 00:51:34.617128 13766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d43ea16e7e04c4397a21fb875dc6c11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:34.617238 13553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d43ea16e7e04c4397a21fb875dc6c11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:34.622442 14122 tablet_bootstrap.cc:492] T 1d43ea16e7e04c4397a21fb875dc6c11 P 5c065702f62545968a5cf6d05ff45a29: Bootstrap starting.
I20260810 00:51:34.622442 14125 tablet_bootstrap.cc:492] T 1d43ea16e7e04c4397a21fb875dc6c11 P 909baa319b8e46f7bf50d9a57fe1da8f: Bootstrap starting.
I20260810 00:51:34.622575 13986 tablet_service.cc:1467] Processing CreateTablet for tablet 1d43ea16e7e04c4397a21fb875dc6c11 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=20c33900d26c4d7f926299ded665a5fe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 00:51:34.623390 13986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d43ea16e7e04c4397a21fb875dc6c11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:34.623883 14122 tablet_bootstrap.cc:654] T 1d43ea16e7e04c4397a21fb875dc6c11 P 5c065702f62545968a5cf6d05ff45a29: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:34.624032 14125 tablet_bootstrap.cc:654] T 1d43ea16e7e04c4397a21fb875dc6c11 P 909baa319b8e46f7bf50d9a57fe1da8f: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:34.626331 14122 tablet_bootstrap.cc:492] T 1d43ea16e7e04c4397a21fb875dc6c11 P 5c065702f62545968a5cf6d05ff45a29: No bootstrap required, opened a new log
I20260810 00:51:34.626468 14122 ts_tablet_manager.cc:1397] T 1d43ea16e7e04c4397a21fb875dc6c11 P 5c065702f62545968a5cf6d05ff45a29: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 00:51:34.627084 14122 raft_consensus.cc:348] T 1d43ea16e7e04c4397a21fb875dc6c11 P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:34.627285 14122 raft_consensus.cc:374] T 1d43ea16e7e04c4397a21fb875dc6c11 P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:34.627372 14122 raft_consensus.cc:729] T 1d43ea16e7e04c4397a21fb875dc6c11 P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c065702f62545968a5cf6d05ff45a29, State: Initialized, Role: FOLLOWER
I20260810 00:51:34.627585 14122 consensus_queue.cc:260] T 1d43ea16e7e04c4397a21fb875dc6c11 P 5c065702f62545968a5cf6d05ff45a29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:34.627998 14130 tablet_replica.cc:439] T 1d43ea16e7e04c4397a21fb875dc6c11 P 5c065702f62545968a5cf6d05ff45a29: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } } }
I20260810 00:51:34.628350 14130 tablet_replica.cc:442] T 1d43ea16e7e04c4397a21fb875dc6c11 P 5c065702f62545968a5cf6d05ff45a29: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:51:34.628800 14122 ts_tablet_manager.cc:1428] T 1d43ea16e7e04c4397a21fb875dc6c11 P 5c065702f62545968a5cf6d05ff45a29: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 00:51:34.630724 14121 tablet_bootstrap.cc:492] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9: Bootstrap starting.
I20260810 00:51:34.632272 14121 tablet_bootstrap.cc:654] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:34.632483 14125 tablet_bootstrap.cc:492] T 1d43ea16e7e04c4397a21fb875dc6c11 P 909baa319b8e46f7bf50d9a57fe1da8f: No bootstrap required, opened a new log
I20260810 00:51:34.632645 14125 ts_tablet_manager.cc:1397] T 1d43ea16e7e04c4397a21fb875dc6c11 P 909baa319b8e46f7bf50d9a57fe1da8f: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.003s
I20260810 00:51:34.633373 14125 raft_consensus.cc:348] T 1d43ea16e7e04c4397a21fb875dc6c11 P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:34.633564 14125 raft_consensus.cc:374] T 1d43ea16e7e04c4397a21fb875dc6c11 P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:34.633659 14125 raft_consensus.cc:729] T 1d43ea16e7e04c4397a21fb875dc6c11 P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 909baa319b8e46f7bf50d9a57fe1da8f, State: Initialized, Role: FOLLOWER
I20260810 00:51:34.633881 14125 consensus_queue.cc:260] T 1d43ea16e7e04c4397a21fb875dc6c11 P 909baa319b8e46f7bf50d9a57fe1da8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:34.634336 14129 tablet_replica.cc:439] T 1d43ea16e7e04c4397a21fb875dc6c11 P 909baa319b8e46f7bf50d9a57fe1da8f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } } }
I20260810 00:51:34.634649 14129 tablet_replica.cc:442] T 1d43ea16e7e04c4397a21fb875dc6c11 P 909baa319b8e46f7bf50d9a57fe1da8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:51:34.634323 14121 tablet_bootstrap.cc:492] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9: No bootstrap required, opened a new log
I20260810 00:51:34.634907 14121 ts_tablet_manager.cc:1397] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 00:51:34.634991 14125 ts_tablet_manager.cc:1428] T 1d43ea16e7e04c4397a21fb875dc6c11 P 909baa319b8e46f7bf50d9a57fe1da8f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 00:51:34.635613 14121 raft_consensus.cc:348] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:34.635795 14121 raft_consensus.cc:374] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:34.635922 14121 raft_consensus.cc:729] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22d1fe55e79e41a9b2fcef79196dcdb9, State: Initialized, Role: FOLLOWER
I20260810 00:51:34.636153 14121 consensus_queue.cc:260] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:34.636611 14128 tablet_replica.cc:439] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } } }
I20260810 00:51:34.636766 14121 ts_tablet_manager.cc:1428] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 00:51:34.636916 14128 tablet_replica.cc:442] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:51:34.925918 14128 raft_consensus.cc:482] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:51:34.926100 14128 raft_consensus.cc:504] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:34.926720 14128 leader_election.cc:288] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 909baa319b8e46f7bf50d9a57fe1da8f (127.10.97.65:42495), 5c065702f62545968a5cf6d05ff45a29 (127.10.97.66:35733)
I20260810 00:51:34.927275 13595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d43ea16e7e04c4397a21fb875dc6c11" candidate_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" is_pre_election: true
I20260810 00:51:34.927317 13803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d43ea16e7e04c4397a21fb875dc6c11" candidate_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c065702f62545968a5cf6d05ff45a29" is_pre_election: true
I20260810 00:51:34.927561 13803 raft_consensus.cc:2454] T 1d43ea16e7e04c4397a21fb875dc6c11 P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22d1fe55e79e41a9b2fcef79196dcdb9 in term 0.
I20260810 00:51:34.927562 13595 raft_consensus.cc:2454] T 1d43ea16e7e04c4397a21fb875dc6c11 P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22d1fe55e79e41a9b2fcef79196dcdb9 in term 0.
I20260810 00:51:34.927995 13899 leader_election.cc:302] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22d1fe55e79e41a9b2fcef79196dcdb9, 5c065702f62545968a5cf6d05ff45a29; no voters:
I20260810 00:51:34.928269 14128 raft_consensus.cc:2781] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:51:34.928357 14128 raft_consensus.cc:482] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:51:34.928457 14128 raft_consensus.cc:3037] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:34.930344 14128 raft_consensus.cc:504] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:34.930820 14128 leader_election.cc:288] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 [CANDIDATE]: Term 1 election: Requested vote from peers 909baa319b8e46f7bf50d9a57fe1da8f (127.10.97.65:42495), 5c065702f62545968a5cf6d05ff45a29 (127.10.97.66:35733)
I20260810 00:51:34.931257 13595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d43ea16e7e04c4397a21fb875dc6c11" candidate_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "909baa319b8e46f7bf50d9a57fe1da8f"
I20260810 00:51:34.931293 13803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d43ea16e7e04c4397a21fb875dc6c11" candidate_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c065702f62545968a5cf6d05ff45a29"
I20260810 00:51:34.931478 13803 raft_consensus.cc:3037] T 1d43ea16e7e04c4397a21fb875dc6c11 P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:34.931478 13595 raft_consensus.cc:3037] T 1d43ea16e7e04c4397a21fb875dc6c11 P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:34.934295 13595 raft_consensus.cc:2454] T 1d43ea16e7e04c4397a21fb875dc6c11 P 909baa319b8e46f7bf50d9a57fe1da8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d1fe55e79e41a9b2fcef79196dcdb9 in term 1.
I20260810 00:51:34.934296 13803 raft_consensus.cc:2454] T 1d43ea16e7e04c4397a21fb875dc6c11 P 5c065702f62545968a5cf6d05ff45a29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d1fe55e79e41a9b2fcef79196dcdb9 in term 1.
I20260810 00:51:34.934659 13899 leader_election.cc:302] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22d1fe55e79e41a9b2fcef79196dcdb9, 5c065702f62545968a5cf6d05ff45a29; no voters:
I20260810 00:51:34.934866 14128 raft_consensus.cc:2781] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:34.935014 14128 raft_consensus.cc:686] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 LEADER]: Becoming Leader. State: Replica: 22d1fe55e79e41a9b2fcef79196dcdb9, State: Running, Role: LEADER
I20260810 00:51:34.935238 14128 consensus_queue.cc:237] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:34.936765 14136 tablet_replica.cc:439] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9: TxnStatusTablet state changed. Reason: New leader 22d1fe55e79e41a9b2fcef79196dcdb9. Latest consensus state: current_term: 1 leader_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } } }
I20260810 00:51:34.937011 14136 tablet_replica.cc:442] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:51:34.937531 14155 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 00:51:34.937816 14155 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 00:51:34.937959 13353 catalog_manager.cc:5495] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22d1fe55e79e41a9b2fcef79196dcdb9 (127.10.97.67). New cstate: current_term: 1 leader_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } health_report { overall_health: HEALTHY } } }
I20260810 00:51:34.989214 14147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:34.982993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:39951 (local address 127.0.0.1:59230)
0810 00:51:34.983234 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:34.983257 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:34.983403 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:34.984323 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:34.984328 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:34.984341 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:51:34.984513 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:34.984522 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.985645 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.985650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:34.986657 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:34.986673 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.987643 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.987647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:34.987768 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:34.987858 (+ 90us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:51:34.988817 (+ 959us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:51:34.988827 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 00:51:34.988873 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 00:51:34.988989 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 00:51:34.989356 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:34.983067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59230 (local address 127.10.97.67:39951)
0810 00:51:34.983978 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:34.983982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:34.983997 (+ 15us) server_negotiation.cc:407] Connection header received
0810 00:51:34.984065 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:34.984068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:34.984149 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:34.984283 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:51:34.984671 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.985498 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.986802 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:34.987511 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:34.987646 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:34.988071 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:51:34.988725 (+ 654us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:51:34.988862 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:34.988923 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 00:51:34.989005 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":810,"thread_start_us":656,"threads_started":1}
I20260810 00:51:34.996874 13803 raft_consensus.cc:1261] T 1d43ea16e7e04c4397a21fb875dc6c11 P 5c065702f62545968a5cf6d05ff45a29 [term 1 FOLLOWER]: Refusing update from remote peer 22d1fe55e79e41a9b2fcef79196dcdb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:51:34.996933 13595 raft_consensus.cc:1261] T 1d43ea16e7e04c4397a21fb875dc6c11 P 909baa319b8e46f7bf50d9a57fe1da8f [term 1 FOLLOWER]: Refusing update from remote peer 22d1fe55e79e41a9b2fcef79196dcdb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:51:34.997702 14128 consensus_queue.cc:1035] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:51:34.998041 14136 consensus_queue.cc:1035] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:51:35.004665 14130 tablet_replica.cc:439] T 1d43ea16e7e04c4397a21fb875dc6c11 P 5c065702f62545968a5cf6d05ff45a29: TxnStatusTablet state changed. Reason: New leader 22d1fe55e79e41a9b2fcef79196dcdb9. Latest consensus state: current_term: 1 leader_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } } }
I20260810 00:51:35.004935 14130 tablet_replica.cc:442] T 1d43ea16e7e04c4397a21fb875dc6c11 P 5c065702f62545968a5cf6d05ff45a29: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:51:35.007333 14159 mvcc.cc:205] Tried to move back new op lower bound from 7316779397098119168 to 7316779396856877056. Current Snapshot: MvccSnapshot[applied={T|T < 7316779397098119168}]
I20260810 00:51:35.008354 14128 tablet_replica.cc:439] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } } }
I20260810 00:51:35.008618 14128 tablet_replica.cc:442] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:51:35.012800 14130 tablet_replica.cc:439] T 1d43ea16e7e04c4397a21fb875dc6c11 P 5c065702f62545968a5cf6d05ff45a29: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } } }
I20260810 00:51:35.013010 14130 tablet_replica.cc:442] T 1d43ea16e7e04c4397a21fb875dc6c11 P 5c065702f62545968a5cf6d05ff45a29: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:51:35.014940 14129 tablet_replica.cc:439] T 1d43ea16e7e04c4397a21fb875dc6c11 P 909baa319b8e46f7bf50d9a57fe1da8f: TxnStatusTablet state changed. Reason: New leader 22d1fe55e79e41a9b2fcef79196dcdb9. Latest consensus state: current_term: 1 leader_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } } }
I20260810 00:51:35.015214 14129 tablet_replica.cc:442] T 1d43ea16e7e04c4397a21fb875dc6c11 P 909baa319b8e46f7bf50d9a57fe1da8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:51:35.017879 14129 tablet_replica.cc:439] T 1d43ea16e7e04c4397a21fb875dc6c11 P 909baa319b8e46f7bf50d9a57fe1da8f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } } }
I20260810 00:51:35.018110 14129 tablet_replica.cc:442] T 1d43ea16e7e04c4397a21fb875dc6c11 P 909baa319b8e46f7bf50d9a57fe1da8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:51:35.018847 14136 tablet_replica.cc:439] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } } }
I20260810 00:51:35.019268 14136 tablet_replica.cc:442] T 1d43ea16e7e04c4397a21fb875dc6c11 P 22d1fe55e79e41a9b2fcef79196dcdb9: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:51:35.025022 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:35.019583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59236 (local address 127.10.97.67:39951)
0810 00:51:35.019954 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:35.019959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:35.020473 (+ 514us) server_negotiation.cc:407] Connection header received
0810 00:51:35.020594 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:35.020599 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:35.020683 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:35.020780 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:51:35.021117 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.022191 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.023432 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.023888 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.023998 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:35.024384 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:51:35.024476 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:51:35.024554 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:35.024689 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 00:51:35.024771 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262}
I20260810 00:51:35.038106 14136 consensus_queue.cc:1035] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:51:35.039645 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:35.033715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:42579 (local address 127.10.97.67:39951)
0810 00:51:35.033884 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:35.033893 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:35.034228 (+ 335us) server_negotiation.cc:407] Connection header received
0810 00:51:35.034367 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:35.034372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:35.034450 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:35.034539 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:51:35.034952 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.036122 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.037665 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.038225 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.038343 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:35.038918 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:51:35.039019 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:51:35.039109 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:35.039272 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 00:51:35.039344 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":12,"server-negotiator.queue_time_us":68}
I20260810 00:51:35.040355 14181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:35.033617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:39951 (local address 127.10.97.67:42579)
0810 00:51:35.034131 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:35.034151 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:35.034260 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:35.034558 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:35.034560 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:35.034569 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:51:35.034810 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:35.034816 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.036237 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.036240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:35.037549 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:35.037561 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.038491 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.038497 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:35.038668 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:35.038731 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:51:35.039154 (+ 423us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:51:35.039162 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 00:51:35.039794 (+ 632us) client_negotiation.cc:240] Negotiation successful
0810 00:51:35.039938 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":240,"threads_started":1}
I20260810 00:51:35.046370 14128 consensus_queue.cc:1035] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:51:35.100196 13355 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:51:35.100436 13355 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:51:35.103595 13355 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc: Sending DeleteTablet for 3 replicas of tablet 8873359c84814724a0efed985064a0a3
I20260810 00:51:35.104553 13986 tablet_service.cc:1514] Processing DeleteTablet for tablet 8873359c84814724a0efed985064a0a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:35 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:59228
I20260810 00:51:35.104831 13766 tablet_service.cc:1514] Processing DeleteTablet for tablet 8873359c84814724a0efed985064a0a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:35 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:51508
I20260810 00:51:35.106122 13355 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:51:35.106763 13355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:51:35.108278 13553 tablet_service.cc:1514] Processing DeleteTablet for tablet 8873359c84814724a0efed985064a0a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:35 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:45054
I20260810 00:51:35.115629 14188 tablet_replica.cc:330] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9: stopping tablet replica
I20260810 00:51:35.116253 14188 raft_consensus.cc:2229] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:51:35.117025 13986 tablet_service.cc:1467] Processing CreateTablet for tablet 48aa6cf3655b4970b429245c2324789f (DEFAULT_TABLE table=test-table [id=b1a7f4ba5d4742ac8a2f326f4c96d913]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:35.117134 13765 tablet_service.cc:1467] Processing CreateTablet for tablet 48aa6cf3655b4970b429245c2324789f (DEFAULT_TABLE table=test-table [id=b1a7f4ba5d4742ac8a2f326f4c96d913]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:35.117479 13986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48aa6cf3655b4970b429245c2324789f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:35.117700 13765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48aa6cf3655b4970b429245c2324789f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:35.117616 13552 tablet_service.cc:1467] Processing CreateTablet for tablet 48aa6cf3655b4970b429245c2324789f (DEFAULT_TABLE table=test-table [id=b1a7f4ba5d4742ac8a2f326f4c96d913]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:35.118119 13552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48aa6cf3655b4970b429245c2324789f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:35.123641 14121 tablet_bootstrap.cc:492] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9: Bootstrap starting.
I20260810 00:51:35.124830 14189 tablet_replica.cc:330] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29: stopping tablet replica
I20260810 00:51:35.125272 14121 tablet_bootstrap.cc:654] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:35.125269 14189 raft_consensus.cc:2229] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:51:35.125692 14189 raft_consensus.cc:2258] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:35.126293 14191 tablet_replica.cc:330] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f: stopping tablet replica
I20260810 00:51:35.126662 14191 raft_consensus.cc:2229] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:51:35.127007 14191 raft_consensus.cc:2258] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:35.127002 14188 raft_consensus.cc:2258] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:35.128038 14125 tablet_bootstrap.cc:492] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f: Bootstrap starting.
I20260810 00:51:35.129024 14191 ts_tablet_manager.cc:1905] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:35.128942 14189 ts_tablet_manager.cc:1905] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:35.129720 14125 tablet_bootstrap.cc:654] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:35.130808 14188 ts_tablet_manager.cc:1905] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:35.135164 14121 tablet_bootstrap.cc:492] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9: No bootstrap required, opened a new log
I20260810 00:51:35.135353 14121 ts_tablet_manager.cc:1397] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20260810 00:51:35.136119 14121 raft_consensus.cc:348] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:35.136369 14121 raft_consensus.cc:374] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:35.136442 14121 raft_consensus.cc:729] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22d1fe55e79e41a9b2fcef79196dcdb9, State: Initialized, Role: FOLLOWER
I20260810 00:51:35.136668 14121 consensus_queue.cc:260] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:35.137301 14121 ts_tablet_manager.cc:1428] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 00:51:35.137728 14191 ts_tablet_manager.cc:1918] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:51:35.137887 14191 log.cc:1196] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/8873359c84814724a0efed985064a0a3
I20260810 00:51:35.138393 14191 ts_tablet_manager.cc:1939] T 8873359c84814724a0efed985064a0a3 P 909baa319b8e46f7bf50d9a57fe1da8f: Deleting consensus metadata
I20260810 00:51:35.139375 14189 ts_tablet_manager.cc:1918] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:51:35.139533 14189 log.cc:1196] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/8873359c84814724a0efed985064a0a3
I20260810 00:51:35.140129 14189 ts_tablet_manager.cc:1939] T 8873359c84814724a0efed985064a0a3 P 5c065702f62545968a5cf6d05ff45a29: Deleting consensus metadata
I20260810 00:51:35.140519 13329 catalog_manager.cc:4841] TS 909baa319b8e46f7bf50d9a57fe1da8f (127.10.97.65:42495): tablet 8873359c84814724a0efed985064a0a3 (table test-table [id=5d81b554db2748c695190de8eddebcc1]) successfully deleted
I20260810 00:51:35.141891 13329 catalog_manager.cc:4841] TS 5c065702f62545968a5cf6d05ff45a29 (127.10.97.66:35733): tablet 8873359c84814724a0efed985064a0a3 (table test-table [id=5d81b554db2748c695190de8eddebcc1]) successfully deleted
I20260810 00:51:35.144583 14188 ts_tablet_manager.cc:1918] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:51:35.144743 14188 log.cc:1196] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/8873359c84814724a0efed985064a0a3
I20260810 00:51:35.145155 14188 ts_tablet_manager.cc:1939] T 8873359c84814724a0efed985064a0a3 P 22d1fe55e79e41a9b2fcef79196dcdb9: Deleting consensus metadata
I20260810 00:51:35.146360 13332 catalog_manager.cc:4841] TS 22d1fe55e79e41a9b2fcef79196dcdb9 (127.10.97.67:39951): tablet 8873359c84814724a0efed985064a0a3 (table test-table [id=5d81b554db2748c695190de8eddebcc1]) successfully deleted
I20260810 00:51:35.146740 14125 tablet_bootstrap.cc:492] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f: No bootstrap required, opened a new log
I20260810 00:51:35.146891 14125 ts_tablet_manager.cc:1397] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.006s
I20260810 00:51:35.147433 14125 raft_consensus.cc:348] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:35.147624 14125 raft_consensus.cc:374] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:35.147702 14125 raft_consensus.cc:729] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 909baa319b8e46f7bf50d9a57fe1da8f, State: Initialized, Role: FOLLOWER
I20260810 00:51:35.147845 14195 tablet_bootstrap.cc:492] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29: Bootstrap starting.
I20260810 00:51:35.148027 14125 consensus_queue.cc:260] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:35.149133 14129 raft_consensus.cc:482] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:51:35.149411 14195 tablet_bootstrap.cc:654] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:35.149447 14125 ts_tablet_manager.cc:1428] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 00:51:35.149339 14129 raft_consensus.cc:504] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:35.151093 14129 leader_election.cc:288] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c065702f62545968a5cf6d05ff45a29 (127.10.97.66:35733), 22d1fe55e79e41a9b2fcef79196dcdb9 (127.10.97.67:39951)
I20260810 00:51:35.151932 14195 tablet_bootstrap.cc:492] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29: No bootstrap required, opened a new log
I20260810 00:51:35.152878 14195 ts_tablet_manager.cc:1397] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260810 00:51:35.153685 14195 raft_consensus.cc:348] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:35.153999 14195 raft_consensus.cc:374] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:35.154140 14195 raft_consensus.cc:729] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c065702f62545968a5cf6d05ff45a29, State: Initialized, Role: FOLLOWER
I20260810 00:51:35.154479 14195 consensus_queue.cc:260] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:35.155086 14195 ts_tablet_manager.cc:1428] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 00:51:35.162966 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:35.152188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:58735 (local address 127.10.97.66:35733)
0810 00:51:35.156264 (+ 4076us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:35.156270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:35.156292 (+ 22us) server_negotiation.cc:407] Connection header received
0810 00:51:35.156348 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:35.156353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:35.156422 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:35.157150 (+ 728us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:35.157415 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.158572 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.160636 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.161861 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.162031 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:35.162105 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:35.162510 (+ 405us) server_negotiation.cc:299] Negotiation successful
0810 00:51:35.162650 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3957,"thread_start_us":7456,"threads_started":1}
I20260810 00:51:35.163003 14199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:35.151640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:35733 (local address 127.10.97.65:58735)
0810 00:51:35.152416 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:35.152440 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:35.152561 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:35.156572 (+ 4011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:35.156577 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:35.156610 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:35.156974 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:35.156991 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.158741 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.158747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:35.160467 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:35.160480 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.162141 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.162152 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:35.162337 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:35.162373 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 00:51:35.162475 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 00:51:35.162602 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":19,"thread_start_us":1115,"threads_started":1}
I20260810 00:51:35.163729 13802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48aa6cf3655b4970b429245c2324789f" candidate_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c065702f62545968a5cf6d05ff45a29" is_pre_election: true
I20260810 00:51:35.164077 13802 raft_consensus.cc:2454] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 909baa319b8e46f7bf50d9a57fe1da8f in term 0.
I20260810 00:51:35.164386 14201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:35.152122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:39951 (local address 127.10.97.65:59503)
0810 00:51:35.155686 (+ 3564us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:35.155709 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:35.155852 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:35.157560 (+ 1708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:35.157565 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:35.157592 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:35.157970 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:35.157983 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.159438 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.159442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:35.161732 (+ 2290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:35.161743 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.163571 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.163578 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:35.163745 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:35.163778 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 00:51:35.163924 (+ 146us) client_negotiation.cc:240] Negotiation successful
0810 00:51:35.164047 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3434,"mutex_wait_us":3,"thread_start_us":4288,"threads_started":1}
I20260810 00:51:35.164705 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:35.152550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:59503 (local address 127.10.97.67:39951)
0810 00:51:35.152739 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:35.152745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:35.155793 (+ 3048us) server_negotiation.cc:407] Connection header received
0810 00:51:35.156061 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:35.156066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:35.156147 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:35.156238 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:35.158175 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.159238 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.161876 (+ 2638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.163239 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.163374 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:35.163422 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:35.164055 (+ 633us) server_negotiation.cc:299] Negotiation successful
0810 00:51:35.164259 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 00:51:35.164970 13489 leader_election.cc:302] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c065702f62545968a5cf6d05ff45a29, 909baa319b8e46f7bf50d9a57fe1da8f; no voters:
I20260810 00:51:35.165395 14129 raft_consensus.cc:2781] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:51:35.165527 14129 raft_consensus.cc:482] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:51:35.165608 14129 raft_consensus.cc:3037] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:35.165661 14013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48aa6cf3655b4970b429245c2324789f" candidate_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" is_pre_election: true
I20260810 00:51:35.166101 14013 raft_consensus.cc:2454] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 909baa319b8e46f7bf50d9a57fe1da8f in term 0.
I20260810 00:51:35.168596 14129 raft_consensus.cc:504] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:35.169245 14129 leader_election.cc:288] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [CANDIDATE]: Term 1 election: Requested vote from peers 5c065702f62545968a5cf6d05ff45a29 (127.10.97.66:35733), 22d1fe55e79e41a9b2fcef79196dcdb9 (127.10.97.67:39951)
I20260810 00:51:35.169687 13802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48aa6cf3655b4970b429245c2324789f" candidate_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c065702f62545968a5cf6d05ff45a29"
I20260810 00:51:35.169785 14013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48aa6cf3655b4970b429245c2324789f" candidate_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9"
I20260810 00:51:35.169873 13802 raft_consensus.cc:3037] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:35.170007 14013 raft_consensus.cc:3037] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:35.172266 14013 raft_consensus.cc:2454] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 909baa319b8e46f7bf50d9a57fe1da8f in term 1.
I20260810 00:51:35.172631 13491 leader_election.cc:302] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22d1fe55e79e41a9b2fcef79196dcdb9, 909baa319b8e46f7bf50d9a57fe1da8f; no voters:
I20260810 00:51:35.172752 13802 raft_consensus.cc:2454] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 909baa319b8e46f7bf50d9a57fe1da8f in term 1.
I20260810 00:51:35.172917 14129 raft_consensus.cc:2781] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:35.173401 14129 raft_consensus.cc:686] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [term 1 LEADER]: Becoming Leader. State: Replica: 909baa319b8e46f7bf50d9a57fe1da8f, State: Running, Role: LEADER
I20260810 00:51:35.173758 14129 consensus_queue.cc:237] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:35.177160 13355 catalog_manager.cc:5495] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 909baa319b8e46f7bf50d9a57fe1da8f (127.10.97.65). New cstate: current_term: 1 leader_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } health_report { overall_health: UNKNOWN } } }
I20260810 00:51:35.207101 14205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:35.201054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45068 (local address 127.10.97.65:42495)
0810 00:51:35.201697 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:35.201703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:35.201725 (+ 22us) server_negotiation.cc:407] Connection header received
0810 00:51:35.201810 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:35.201815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:35.201905 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:35.202070 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:51:35.202373 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.203504 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.204770 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.205402 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.205518 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:35.205865 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:51:35.206530 (+ 665us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:51:35.206660 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:35.206701 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 00:51:35.206772 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":364,"threads_started":1}
I20260810 00:51:35.221446 14208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:35.215882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:39951 (local address 127.10.97.65:56711)
0810 00:51:35.216418 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:35.216433 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:35.216536 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:35.216835 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:35.216838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:35.216846 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:51:35.217081 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:35.217088 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.218447 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.218450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:35.219453 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:35.219463 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.220466 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.220470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:35.220586 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:35.220636 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:51:35.221027 (+ 391us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:51:35.221033 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 00:51:35.221082 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 00:51:35.221176 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":253,"threads_started":1}
I20260810 00:51:35.221489 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:35.215994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:56711 (local address 127.10.97.67:39951)
0810 00:51:35.216149 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:35.216155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:35.216512 (+ 357us) server_negotiation.cc:407] Connection header received
0810 00:51:35.216645 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:35.216651 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:35.216733 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:35.216833 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:51:35.217227 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.218337 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.219592 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.220309 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.220442 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:35.220800 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:51:35.220938 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:51:35.221034 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:35.221121 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 00:51:35.221210 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 00:51:35.230304 13802 raft_consensus.cc:1261] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29 [term 1 FOLLOWER]: Refusing update from remote peer 909baa319b8e46f7bf50d9a57fe1da8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:51:35.230501 14013 raft_consensus.cc:1261] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 FOLLOWER]: Refusing update from remote peer 909baa319b8e46f7bf50d9a57fe1da8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:51:35.230865 14203 consensus_queue.cc:1035] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:51:35.231169 14129 consensus_queue.cc:1035] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:51:35.238601 14163 mvcc.cc:205] Tried to move back new op lower bound from 7316779398058164224 to 7316779397834964992. Current Snapshot: MvccSnapshot[applied={T|T < 7316779398058164224}]
I20260810 00:51:35.261276 14205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:35.255757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:47997 (local address 127.10.97.65:42495)
0810 00:51:35.255910 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:35.255916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:35.255932 (+ 16us) server_negotiation.cc:407] Connection header received
0810 00:51:35.255988 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:35.255992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:35.256062 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:35.256583 (+ 521us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:51:35.256728 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.257644 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.259067 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.259655 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.259780 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:35.260406 (+ 626us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:51:35.260670 (+ 264us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:51:35.260751 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:35.260891 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 00:51:35.260961 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 00:51:35.261307 14181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:35.255178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42495 (local address 127.10.97.67:47997)
0810 00:51:35.255362 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:35.255385 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:35.255496 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:35.256219 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:35.256223 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:35.256236 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:51:35.256445 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:35.256457 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.257772 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.257777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:35.258940 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:35.258952 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:35.259841 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:35.259847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:35.260047 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:35.260101 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:51:35.260769 (+ 668us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:51:35.260784 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 00:51:35.260869 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 00:51:35.261002 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 00:51:35.262579 14163 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 00:51:36.852471 13355 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:51:36.852752 13355 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:51:36.856488 13355 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc: Sending DeleteTablet for 3 replicas of tablet 48aa6cf3655b4970b429245c2324789f
I20260810 00:51:36.857141 13552 tablet_service.cc:1514] Processing DeleteTablet for tablet 48aa6cf3655b4970b429245c2324789f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:36 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:45054
I20260810 00:51:36.857209 13765 tablet_service.cc:1514] Processing DeleteTablet for tablet 48aa6cf3655b4970b429245c2324789f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:36 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:51508
I20260810 00:51:36.857700 14252 tablet_replica.cc:330] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f: stopping tablet replica
I20260810 00:51:36.857708 13986 tablet_service.cc:1514] Processing DeleteTablet for tablet 48aa6cf3655b4970b429245c2324789f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:36 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:59228
I20260810 00:51:36.858003 14252 raft_consensus.cc:2229] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:51:36.858588 14252 raft_consensus.cc:2258] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:36.859648 14252 ts_tablet_manager.cc:1905] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:36.866792 14252 ts_tablet_manager.cc:1918] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 00:51:36.867115 14252 log.cc:1196] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/48aa6cf3655b4970b429245c2324789f
I20260810 00:51:36.868551 14255 tablet_replica.cc:330] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9: stopping tablet replica
I20260810 00:51:36.868750 14255 raft_consensus.cc:2229] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:51:36.868981 14255 raft_consensus.cc:2258] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:36.869975 14255 ts_tablet_manager.cc:1905] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:36.870447 14252 ts_tablet_manager.cc:1939] T 48aa6cf3655b4970b429245c2324789f P 909baa319b8e46f7bf50d9a57fe1da8f: Deleting consensus metadata
I20260810 00:51:36.871548 13329 catalog_manager.cc:4841] TS 909baa319b8e46f7bf50d9a57fe1da8f (127.10.97.65:42495): tablet 48aa6cf3655b4970b429245c2324789f (table test-table [id=b1a7f4ba5d4742ac8a2f326f4c96d913]) successfully deleted
I20260810 00:51:36.874869 14255 ts_tablet_manager.cc:1918] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 00:51:36.874987 14255 log.cc:1196] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/48aa6cf3655b4970b429245c2324789f
I20260810 00:51:36.875371 14255 ts_tablet_manager.cc:1939] T 48aa6cf3655b4970b429245c2324789f P 22d1fe55e79e41a9b2fcef79196dcdb9: Deleting consensus metadata
I20260810 00:51:36.876219 13332 catalog_manager.cc:4841] TS 22d1fe55e79e41a9b2fcef79196dcdb9 (127.10.97.67:39951): tablet 48aa6cf3655b4970b429245c2324789f (table test-table [id=b1a7f4ba5d4742ac8a2f326f4c96d913]) successfully deleted
I20260810 00:51:36.877849 14254 tablet_replica.cc:330] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29: stopping tablet replica
I20260810 00:51:36.878017 14254 raft_consensus.cc:2229] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:51:36.878226 14254 raft_consensus.cc:2258] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:36.878925 14254 ts_tablet_manager.cc:1905] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:36.883111 14254 ts_tablet_manager.cc:1918] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 00:51:36.883260 14254 log.cc:1196] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/48aa6cf3655b4970b429245c2324789f
I20260810 00:51:36.883961 14254 ts_tablet_manager.cc:1939] T 48aa6cf3655b4970b429245c2324789f P 5c065702f62545968a5cf6d05ff45a29: Deleting consensus metadata
I20260810 00:51:36.884871 13329 catalog_manager.cc:4841] TS 5c065702f62545968a5cf6d05ff45a29 (127.10.97.66:35733): tablet 48aa6cf3655b4970b429245c2324789f (table test-table [id=b1a7f4ba5d4742ac8a2f326f4c96d913]) successfully deleted
Aug 10 00:51:36 dist-test-slave-jlzn krb5kdc[13289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323096, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 00:51:36 dist-test-slave-jlzn krb5kdc[13289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323096, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:36.916996 14269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:36.904100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37512 (local address 127.10.97.126:38495)
0810 00:51:36.904886 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:36.904891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:36.904910 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:51:36.904957 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:36.904960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:36.905028 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:36.905111 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:36.906294 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:36.907074 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:36.908330 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:36.908746 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:36.908833 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:36.912331 (+ 3498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:36.912336 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:36.912340 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:36.912369 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:36.914895 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:36.915441 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:36.915444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:36.915447 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:36.915557 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:36.915833 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:36.915837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:36.915840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:36.916095 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:36.916271 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:36.916502 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 00:51:36.916667 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":534,"threads_started":1}
I20260810 00:51:36.919955 13355 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:51:36.920553 13355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:51:36.926385 13552 tablet_service.cc:1467] Processing CreateTablet for tablet fa391951fd534cfc9ed8bd8167a4831e (DEFAULT_TABLE table=test-table [id=cabc2a06363d417b939bedf92b665aca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:36.926491 13765 tablet_service.cc:1467] Processing CreateTablet for tablet fa391951fd534cfc9ed8bd8167a4831e (DEFAULT_TABLE table=test-table [id=cabc2a06363d417b939bedf92b665aca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:36.926841 13552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa391951fd534cfc9ed8bd8167a4831e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:36.926761 13986 tablet_service.cc:1467] Processing CreateTablet for tablet fa391951fd534cfc9ed8bd8167a4831e (DEFAULT_TABLE table=test-table [id=cabc2a06363d417b939bedf92b665aca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:36.926923 13765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa391951fd534cfc9ed8bd8167a4831e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:36.927167 13986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa391951fd534cfc9ed8bd8167a4831e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:36.931773 14270 tablet_bootstrap.cc:492] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29: Bootstrap starting.
I20260810 00:51:36.933548 14273 tablet_bootstrap.cc:492] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9: Bootstrap starting.
I20260810 00:51:36.933679 14270 tablet_bootstrap.cc:654] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:36.935184 14273 tablet_bootstrap.cc:654] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:36.935936 14271 tablet_bootstrap.cc:492] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f: Bootstrap starting.
I20260810 00:51:36.935999 14270 tablet_bootstrap.cc:492] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29: No bootstrap required, opened a new log
I20260810 00:51:36.936121 14270 ts_tablet_manager.cc:1397] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260810 00:51:36.936838 14270 raft_consensus.cc:348] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:36.937012 14270 raft_consensus.cc:374] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:36.937106 14270 raft_consensus.cc:729] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c065702f62545968a5cf6d05ff45a29, State: Initialized, Role: FOLLOWER
I20260810 00:51:36.937439 14271 tablet_bootstrap.cc:654] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:36.937342 14270 consensus_queue.cc:260] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:36.937680 14273 tablet_bootstrap.cc:492] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9: No bootstrap required, opened a new log
I20260810 00:51:36.937824 14273 ts_tablet_manager.cc:1397] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 00:51:36.938205 14270 ts_tablet_manager.cc:1428] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 00:51:36.938563 14273 raft_consensus.cc:348] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:36.938804 14273 raft_consensus.cc:374] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:36.938895 14273 raft_consensus.cc:729] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22d1fe55e79e41a9b2fcef79196dcdb9, State: Initialized, Role: FOLLOWER
I20260810 00:51:36.939121 14273 consensus_queue.cc:260] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:36.939987 14271 tablet_bootstrap.cc:492] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f: No bootstrap required, opened a new log
I20260810 00:51:36.940184 14271 ts_tablet_manager.cc:1397] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 00:51:36.940750 14273 ts_tablet_manager.cc:1428] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 00:51:36.940919 14271 raft_consensus.cc:348] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:36.941147 14271 raft_consensus.cc:374] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:36.941344 14271 raft_consensus.cc:729] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 909baa319b8e46f7bf50d9a57fe1da8f, State: Initialized, Role: FOLLOWER
I20260810 00:51:36.941586 14271 consensus_queue.cc:260] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:36.942117 14271 ts_tablet_manager.cc:1428] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 00:51:37.053468 14279 raft_consensus.cc:482] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:51:37.053680 14279 raft_consensus.cc:504] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:37.055282 14279 leader_election.cc:288] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 909baa319b8e46f7bf50d9a57fe1da8f (127.10.97.65:42495), 22d1fe55e79e41a9b2fcef79196dcdb9 (127.10.97.67:39951)
I20260810 00:51:37.075246 14287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:37.055570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:57305 (local address 127.10.97.65:42495)
0810 00:51:37.067890 (+ 12320us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:37.067896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:37.067920 (+ 24us) server_negotiation.cc:407] Connection header received
0810 00:51:37.067987 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:37.067991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:37.068061 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:37.068156 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:37.068873 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:37.070019 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:37.072941 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:37.074114 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:37.074284 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:37.074359 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:37.074780 (+ 421us) server_negotiation.cc:299] Negotiation successful
0810 00:51:37.074913 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12172,"thread_start_us":12021,"threads_started":1}
I20260810 00:51:37.075256 14288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:37.055585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:37135 (local address 127.10.97.67:39951)
0810 00:51:37.068009 (+ 12424us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:37.068014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:37.068035 (+ 21us) server_negotiation.cc:407] Connection header received
0810 00:51:37.068094 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:37.068098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:37.068175 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:37.068267 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:37.068992 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:37.070111 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:37.072927 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:37.074170 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:37.074322 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:37.074383 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:37.074780 (+ 397us) server_negotiation.cc:299] Negotiation successful
0810 00:51:37.074913 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12286,"thread_start_us":12118,"threads_started":1}
I20260810 00:51:37.075368 14283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:37.055462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:39951 (local address 127.10.97.66:37135)
0810 00:51:37.067341 (+ 11879us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:37.067358 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:37.067468 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:37.068296 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:37.068300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:37.068331 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:37.068818 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:37.068831 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:37.070223 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:37.070227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:37.072781 (+ 2554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:37.072794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:37.074365 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:37.074371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:37.074580 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:37.074656 (+ 76us) client_negotiation.cc:769] Sending connection context
0810 00:51:37.074752 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 00:51:37.074929 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11745,"mutex_wait_us":236,"thread_start_us":11785,"threads_started":1}
I20260810 00:51:37.075758 14284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:37.055417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42495 (local address 127.10.97.66:57305)
0810 00:51:37.067000 (+ 11583us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:37.067020 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:37.067161 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:37.068215 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:37.068219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:37.068247 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:37.068683 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:37.068701 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:37.070170 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:37.070175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:37.072777 (+ 2602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:37.072794 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:37.074334 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:37.074341 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:37.074578 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:37.074656 (+ 78us) client_negotiation.cc:769] Sending connection context
0810 00:51:37.074752 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 00:51:37.074929 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11439,"mutex_wait_us":193,"thread_start_us":11296,"threads_started":1}
I20260810 00:51:37.076042 14013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa391951fd534cfc9ed8bd8167a4831e" candidate_uuid: "5c065702f62545968a5cf6d05ff45a29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" is_pre_election: true
I20260810 00:51:37.076267 14013 raft_consensus.cc:2454] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c065702f62545968a5cf6d05ff45a29 in term 0.
I20260810 00:51:37.076471 13592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa391951fd534cfc9ed8bd8167a4831e" candidate_uuid: "5c065702f62545968a5cf6d05ff45a29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" is_pre_election: true
I20260810 00:51:37.076700 13592 raft_consensus.cc:2454] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c065702f62545968a5cf6d05ff45a29 in term 0.
I20260810 00:51:37.076798 13691 leader_election.cc:302] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22d1fe55e79e41a9b2fcef79196dcdb9, 5c065702f62545968a5cf6d05ff45a29; no voters:
I20260810 00:51:37.077172 14279 raft_consensus.cc:2781] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:51:37.077272 14279 raft_consensus.cc:482] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:51:37.077363 14279 raft_consensus.cc:3037] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:37.079419 14279 raft_consensus.cc:504] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:37.079973 14279 leader_election.cc:288] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [CANDIDATE]: Term 1 election: Requested vote from peers 909baa319b8e46f7bf50d9a57fe1da8f (127.10.97.65:42495), 22d1fe55e79e41a9b2fcef79196dcdb9 (127.10.97.67:39951)
I20260810 00:51:37.080425 13592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa391951fd534cfc9ed8bd8167a4831e" candidate_uuid: "5c065702f62545968a5cf6d05ff45a29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "909baa319b8e46f7bf50d9a57fe1da8f"
I20260810 00:51:37.080469 14013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa391951fd534cfc9ed8bd8167a4831e" candidate_uuid: "5c065702f62545968a5cf6d05ff45a29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9"
I20260810 00:51:37.080611 13592 raft_consensus.cc:3037] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:37.080684 14013 raft_consensus.cc:3037] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:37.082578 13592 raft_consensus.cc:2454] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c065702f62545968a5cf6d05ff45a29 in term 1.
I20260810 00:51:37.082633 14013 raft_consensus.cc:2454] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c065702f62545968a5cf6d05ff45a29 in term 1.
I20260810 00:51:37.082948 13689 leader_election.cc:302] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c065702f62545968a5cf6d05ff45a29, 909baa319b8e46f7bf50d9a57fe1da8f; no voters:
I20260810 00:51:37.083232 14279 raft_consensus.cc:2781] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:37.083609 14279 raft_consensus.cc:686] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [term 1 LEADER]: Becoming Leader. State: Replica: 5c065702f62545968a5cf6d05ff45a29, State: Running, Role: LEADER
I20260810 00:51:37.084035 14279 consensus_queue.cc:237] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:37.086961 13354 catalog_manager.cc:5495] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c065702f62545968a5cf6d05ff45a29 (127.10.97.66). New cstate: current_term: 1 leader_uuid: "5c065702f62545968a5cf6d05ff45a29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } health_report { overall_health: UNKNOWN } } }
I20260810 00:51:37.131745 14293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:37.123897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51510 (local address 127.10.97.66:35733)
0810 00:51:37.127058 (+ 3161us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:37.127062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:37.127080 (+ 18us) server_negotiation.cc:407] Connection header received
0810 00:51:37.127158 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:37.127161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:37.127225 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:37.127355 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:51:37.127657 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:37.128467 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:37.129665 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:37.130057 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:37.130151 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:37.130585 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:51:37.131187 (+ 602us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:51:37.131295 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:37.131378 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 00:51:37.131435 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3062,"thread_start_us":2961,"threads_started":1}
I20260810 00:51:37.144564 14288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:37.140260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:60099 (local address 127.10.97.67:39951)
0810 00:51:37.140447 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:37.140453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:37.140648 (+ 195us) server_negotiation.cc:407] Connection header received
0810 00:51:37.140782 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:37.140785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:37.140842 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:37.140907 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:51:37.141329 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:37.141971 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:37.143191 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:37.143560 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:37.143642 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:37.144098 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:51:37.144195 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:51:37.144255 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:37.144329 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 00:51:37.144383 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 00:51:37.144667 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:37.140016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:39951 (local address 127.10.97.66:60099)
0810 00:51:37.140570 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:37.140584 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:37.140703 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:37.140920 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:37.140923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:37.140932 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:51:37.141221 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:37.141230 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:37.142066 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:37.142070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:37.143086 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:37.143096 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:37.143736 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:37.143741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:37.143917 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:37.143968 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:51:37.144261 (+ 293us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:51:37.144268 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 00:51:37.144316 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 00:51:37.144403 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":250,"threads_started":1}
I20260810 00:51:37.152122 13592 raft_consensus.cc:1261] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f [term 1 FOLLOWER]: Refusing update from remote peer 5c065702f62545968a5cf6d05ff45a29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:51:37.152266 14013 raft_consensus.cc:1261] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 FOLLOWER]: Refusing update from remote peer 5c065702f62545968a5cf6d05ff45a29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:51:37.152725 14289 consensus_queue.cc:1035] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:51:37.153043 14279 consensus_queue.cc:1035] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:51:37.176931 14293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:37.170624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:33535 (local address 127.10.97.66:35733)
0810 00:51:37.170763 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:37.170769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:37.172285 (+ 1516us) server_negotiation.cc:407] Connection header received
0810 00:51:37.172405 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:37.172408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:37.172462 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:37.172525 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:51:37.172948 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:37.173752 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:37.175171 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:37.175576 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:37.175688 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:37.176135 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:51:37.176371 (+ 236us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:51:37.176477 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:37.176593 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 00:51:37.176669 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 00:51:37.176975 14301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:37.170565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:35733 (local address 127.10.97.67:33535)
0810 00:51:37.172198 (+ 1633us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:37.172212 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:37.172329 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:37.172551 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:37.172556 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:37.172567 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:51:37.172815 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:37.172824 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:37.173880 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:37.173885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:37.175042 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:37.175054 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:37.175738 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:37.175745 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:37.175919 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:37.175963 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:51:37.176485 (+ 522us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:51:37.176495 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 00:51:37.176555 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 00:51:37.176649 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1515,"thread_start_us":1396,"threads_started":1}
I20260810 00:51:37.204972 13354 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:51:37.205248 13354 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:51:37.208108 13354 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc: Sending DeleteTablet for 3 replicas of tablet fa391951fd534cfc9ed8bd8167a4831e
I20260810 00:51:37.208817 13552 tablet_service.cc:1514] Processing DeleteTablet for tablet fa391951fd534cfc9ed8bd8167a4831e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:37 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:45054
I20260810 00:51:37.208986 13765 tablet_service.cc:1514] Processing DeleteTablet for tablet fa391951fd534cfc9ed8bd8167a4831e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:37 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:51508
I20260810 00:51:37.209245 14254 tablet_replica.cc:330] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29: stopping tablet replica
I20260810 00:51:37.209239 14252 tablet_replica.cc:330] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f: stopping tablet replica
I20260810 00:51:37.209435 14252 raft_consensus.cc:2229] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:51:37.209446 14254 raft_consensus.cc:2229] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:51:37.209612 14252 raft_consensus.cc:2258] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:37.209901 14254 raft_consensus.cc:2258] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:37.210898 14252 ts_tablet_manager.cc:1905] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:37.210825 13354 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 00:51:37.211036 14254 ts_tablet_manager.cc:1905] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 00:51:37.211378 13354 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:51:37.217154 14252 ts_tablet_manager.cc:1918] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:51:37.217320 14252 log.cc:1196] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/fa391951fd534cfc9ed8bd8167a4831e
I20260810 00:51:37.217744 14252 ts_tablet_manager.cc:1939] T fa391951fd534cfc9ed8bd8167a4831e P 909baa319b8e46f7bf50d9a57fe1da8f: Deleting consensus metadata
I20260810 00:51:37.218506 13765 tablet_service.cc:1467] Processing CreateTablet for tablet e01226b9c3eb4759a6ed02eabdcbf5b5 (DEFAULT_TABLE table=test-table [id=0748ffc6b2394afbb7e1600e8849fdb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:37.218869 14254 ts_tablet_manager.cc:1918] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:51:37.218915 13765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e01226b9c3eb4759a6ed02eabdcbf5b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:37.219496 13329 catalog_manager.cc:4841] TS 909baa319b8e46f7bf50d9a57fe1da8f (127.10.97.65:42495): tablet fa391951fd534cfc9ed8bd8167a4831e (table test-table [id=cabc2a06363d417b939bedf92b665aca]) successfully deleted
I20260810 00:51:37.219491 13552 tablet_service.cc:1467] Processing CreateTablet for tablet e01226b9c3eb4759a6ed02eabdcbf5b5 (DEFAULT_TABLE table=test-table [id=0748ffc6b2394afbb7e1600e8849fdb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:37.219862 13552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e01226b9c3eb4759a6ed02eabdcbf5b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:37.220682 13986 tablet_service.cc:1514] Processing DeleteTablet for tablet fa391951fd534cfc9ed8bd8167a4831e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:37 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:59228
I20260810 00:51:37.221839 13986 tablet_service.cc:1467] Processing CreateTablet for tablet e01226b9c3eb4759a6ed02eabdcbf5b5 (DEFAULT_TABLE table=test-table [id=0748ffc6b2394afbb7e1600e8849fdb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:37.222218 13986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e01226b9c3eb4759a6ed02eabdcbf5b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:37.225206 14270 tablet_bootstrap.cc:492] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29: Bootstrap starting.
I20260810 00:51:37.225358 14255 tablet_replica.cc:330] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9: stopping tablet replica
I20260810 00:51:37.225546 14255 raft_consensus.cc:2229] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:51:37.225709 14255 raft_consensus.cc:2258] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:37.226166 14271 tablet_bootstrap.cc:492] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f: Bootstrap starting.
I20260810 00:51:37.226790 14270 tablet_bootstrap.cc:654] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:37.227746 14271 tablet_bootstrap.cc:654] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:37.230953 14273 tablet_bootstrap.cc:492] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9: Bootstrap starting.
I20260810 00:51:37.232013 14254 log.cc:1196] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/fa391951fd534cfc9ed8bd8167a4831e
I20260810 00:51:37.232455 14271 tablet_bootstrap.cc:492] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f: No bootstrap required, opened a new log
I20260810 00:51:37.232596 14273 tablet_bootstrap.cc:654] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:37.232677 14271 ts_tablet_manager.cc:1397] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 00:51:37.233202 14254 ts_tablet_manager.cc:1939] T fa391951fd534cfc9ed8bd8167a4831e P 5c065702f62545968a5cf6d05ff45a29: Deleting consensus metadata
I20260810 00:51:37.233389 14271 raft_consensus.cc:348] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:37.233616 14271 raft_consensus.cc:374] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:37.233708 14271 raft_consensus.cc:729] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 909baa319b8e46f7bf50d9a57fe1da8f, State: Initialized, Role: FOLLOWER
I20260810 00:51:37.233860 14255 ts_tablet_manager.cc:1905] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:37.233918 14271 consensus_queue.cc:260] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:37.237519 14271 ts_tablet_manager.cc:1428] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260810 00:51:37.240320 14255 ts_tablet_manager.cc:1918] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:51:37.240540 14255 log.cc:1196] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/fa391951fd534cfc9ed8bd8167a4831e
I20260810 00:51:37.240994 14255 ts_tablet_manager.cc:1939] T fa391951fd534cfc9ed8bd8167a4831e P 22d1fe55e79e41a9b2fcef79196dcdb9: Deleting consensus metadata
I20260810 00:51:37.241824 13332 catalog_manager.cc:4841] TS 22d1fe55e79e41a9b2fcef79196dcdb9 (127.10.97.67:39951): tablet fa391951fd534cfc9ed8bd8167a4831e (table test-table [id=cabc2a06363d417b939bedf92b665aca]) successfully deleted
I20260810 00:51:37.243116 13329 catalog_manager.cc:4841] TS 5c065702f62545968a5cf6d05ff45a29 (127.10.97.66:35733): tablet fa391951fd534cfc9ed8bd8167a4831e (table test-table [id=cabc2a06363d417b939bedf92b665aca]) successfully deleted
I20260810 00:51:37.243508 14270 tablet_bootstrap.cc:492] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29: No bootstrap required, opened a new log
I20260810 00:51:37.243657 14270 ts_tablet_manager.cc:1397] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.000s
I20260810 00:51:37.244247 14270 raft_consensus.cc:348] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:37.244450 14270 raft_consensus.cc:374] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:37.244536 14270 raft_consensus.cc:729] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c065702f62545968a5cf6d05ff45a29, State: Initialized, Role: FOLLOWER
I20260810 00:51:37.244779 14270 consensus_queue.cc:260] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:37.245249 14270 ts_tablet_manager.cc:1428] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 00:51:37.248374 14273 tablet_bootstrap.cc:492] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9: No bootstrap required, opened a new log
I20260810 00:51:37.248528 14273 ts_tablet_manager.cc:1397] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.004s
I20260810 00:51:37.249245 14273 raft_consensus.cc:348] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:37.249487 14273 raft_consensus.cc:374] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:37.249571 14273 raft_consensus.cc:729] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22d1fe55e79e41a9b2fcef79196dcdb9, State: Initialized, Role: FOLLOWER
I20260810 00:51:37.249709 14289 raft_consensus.cc:482] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:51:37.249886 14289 raft_consensus.cc:504] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:37.250568 14289 leader_election.cc:288] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 909baa319b8e46f7bf50d9a57fe1da8f (127.10.97.65:42495), 22d1fe55e79e41a9b2fcef79196dcdb9 (127.10.97.67:39951)
I20260810 00:51:37.250952 13592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01226b9c3eb4759a6ed02eabdcbf5b5" candidate_uuid: "5c065702f62545968a5cf6d05ff45a29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" is_pre_election: true
I20260810 00:51:37.251044 14013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01226b9c3eb4759a6ed02eabdcbf5b5" candidate_uuid: "5c065702f62545968a5cf6d05ff45a29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" is_pre_election: true
I20260810 00:51:37.251176 13592 raft_consensus.cc:2454] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c065702f62545968a5cf6d05ff45a29 in term 0.
I20260810 00:51:37.251564 13689 leader_election.cc:302] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c065702f62545968a5cf6d05ff45a29, 909baa319b8e46f7bf50d9a57fe1da8f; no voters:
I20260810 00:51:37.251482 14273 consensus_queue.cc:260] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:37.251840 14289 raft_consensus.cc:2781] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:51:37.251909 14013 raft_consensus.cc:2454] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c065702f62545968a5cf6d05ff45a29 in term 0.
I20260810 00:51:37.251957 14289 raft_consensus.cc:482] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:51:37.252079 14289 raft_consensus.cc:3037] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:37.254840 14289 raft_consensus.cc:504] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:37.255446 14289 leader_election.cc:288] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [CANDIDATE]: Term 1 election: Requested vote from peers 909baa319b8e46f7bf50d9a57fe1da8f (127.10.97.65:42495), 22d1fe55e79e41a9b2fcef79196dcdb9 (127.10.97.67:39951)
I20260810 00:51:37.255859 14013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01226b9c3eb4759a6ed02eabdcbf5b5" candidate_uuid: "5c065702f62545968a5cf6d05ff45a29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9"
I20260810 00:51:37.255987 13592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01226b9c3eb4759a6ed02eabdcbf5b5" candidate_uuid: "5c065702f62545968a5cf6d05ff45a29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "909baa319b8e46f7bf50d9a57fe1da8f"
I20260810 00:51:37.256062 14013 raft_consensus.cc:3037] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:37.256150 13592 raft_consensus.cc:3037] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:37.257897 14273 ts_tablet_manager.cc:1428] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260810 00:51:37.258755 14013 raft_consensus.cc:2454] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c065702f62545968a5cf6d05ff45a29 in term 1.
I20260810 00:51:37.258755 13592 raft_consensus.cc:2454] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c065702f62545968a5cf6d05ff45a29 in term 1.
I20260810 00:51:37.259176 13691 leader_election.cc:302] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22d1fe55e79e41a9b2fcef79196dcdb9, 5c065702f62545968a5cf6d05ff45a29; no voters:
I20260810 00:51:37.259600 14289 raft_consensus.cc:2781] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:37.259760 14289 raft_consensus.cc:686] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [term 1 LEADER]: Becoming Leader. State: Replica: 5c065702f62545968a5cf6d05ff45a29, State: Running, Role: LEADER
I20260810 00:51:37.260010 14289 consensus_queue.cc:237] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } }
I20260810 00:51:37.262761 13354 catalog_manager.cc:5495] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c065702f62545968a5cf6d05ff45a29 (127.10.97.66). New cstate: current_term: 1 leader_uuid: "5c065702f62545968a5cf6d05ff45a29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c065702f62545968a5cf6d05ff45a29" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 } health_report { overall_health: UNKNOWN } } }
I20260810 00:51:37.286224 14013 raft_consensus.cc:1261] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 FOLLOWER]: Refusing update from remote peer 5c065702f62545968a5cf6d05ff45a29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:51:37.286340 13592 raft_consensus.cc:1261] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f [term 1 FOLLOWER]: Refusing update from remote peer 5c065702f62545968a5cf6d05ff45a29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:51:37.286891 14289 consensus_queue.cc:1035] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "909baa319b8e46f7bf50d9a57fe1da8f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:51:37.287393 14304 consensus_queue.cc:1035] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d1fe55e79e41a9b2fcef79196dcdb9" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 39951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:51:37.290952 14223 mvcc.cc:205] Tried to move back new op lower bound from 7316779406479560704 to 7316779406380974080. Current Snapshot: MvccSnapshot[applied={T|T < 7316779406479560704}]
W20260810 00:51:37.309860 14223 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 00:51:38.900504 13355 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:51:38.900723 13355 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:51:38.904495 13355 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa995c57e6374786acb73c8a5534e7cc: Sending DeleteTablet for 3 replicas of tablet e01226b9c3eb4759a6ed02eabdcbf5b5
I20260810 00:51:38.905184 13552 tablet_service.cc:1514] Processing DeleteTablet for tablet e01226b9c3eb4759a6ed02eabdcbf5b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:38 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:45054
I20260810 00:51:38.905264 13765 tablet_service.cc:1514] Processing DeleteTablet for tablet e01226b9c3eb4759a6ed02eabdcbf5b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:38 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:51508
I20260810 00:51:38.905360 13986 tablet_service.cc:1514] Processing DeleteTablet for tablet e01226b9c3eb4759a6ed02eabdcbf5b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:38 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:59228
I20260810 00:51:38.906030 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 13463
I20260810 00:51:38.920518 14359 tablet_replica.cc:330] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29: stopping tablet replica
I20260810 00:51:38.920729 14359 raft_consensus.cc:2229] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:51:38.921066 14359 raft_consensus.cc:2258] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:38.921579 14360 tablet_replica.cc:330] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9: stopping tablet replica
I20260810 00:51:38.921783 14360 raft_consensus.cc:2229] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:51:38.921993 14360 raft_consensus.cc:2258] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:38.923379 14360 ts_tablet_manager.cc:1905] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:38.924446 14359 ts_tablet_manager.cc:1905] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:38.930759 14360 ts_tablet_manager.cc:1918] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 00:51:38.930970 14360 log.cc:1196] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/e01226b9c3eb4759a6ed02eabdcbf5b5
I20260810 00:51:38.931327 14359 ts_tablet_manager.cc:1918] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 00:51:38.931531 14359 log.cc:1196] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/e01226b9c3eb4759a6ed02eabdcbf5b5
I20260810 00:51:38.931576 14360 ts_tablet_manager.cc:1939] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 22d1fe55e79e41a9b2fcef79196dcdb9: Deleting consensus metadata
I20260810 00:51:38.932047 14359 ts_tablet_manager.cc:1939] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 5c065702f62545968a5cf6d05ff45a29: Deleting consensus metadata
I20260810 00:51:38.932732 13332 catalog_manager.cc:4841] TS 22d1fe55e79e41a9b2fcef79196dcdb9 (127.10.97.67:39951): tablet e01226b9c3eb4759a6ed02eabdcbf5b5 (table test-table [id=0748ffc6b2394afbb7e1600e8849fdb3]) successfully deleted
I20260810 00:51:38.933142 13329 catalog_manager.cc:4841] TS 5c065702f62545968a5cf6d05ff45a29 (127.10.97.66:35733): tablet e01226b9c3eb4759a6ed02eabdcbf5b5 (table test-table [id=0748ffc6b2394afbb7e1600e8849fdb3]) successfully deleted
I20260810 00:51:38.933481 14358 tablet_replica.cc:330] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f: stopping tablet replica
I20260810 00:51:38.933651 14358 raft_consensus.cc:2229] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:51:38.933851 14358 raft_consensus.cc:2258] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:38.934662 14358 ts_tablet_manager.cc:1905] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:38.938862 14358 ts_tablet_manager.cc:1918] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 00:51:38.938979 14358 log.cc:1196] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/e01226b9c3eb4759a6ed02eabdcbf5b5
I20260810 00:51:38.939350 14358 ts_tablet_manager.cc:1939] T e01226b9c3eb4759a6ed02eabdcbf5b5 P 909baa319b8e46f7bf50d9a57fe1da8f: Deleting consensus metadata
Aug 10 00:51:38 dist-test-slave-jlzn krb5kdc[13289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323096, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:51:38.940267 13329 catalog_manager.cc:4841] TS 909baa319b8e46f7bf50d9a57fe1da8f (127.10.97.65:42495): tablet e01226b9c3eb4759a6ed02eabdcbf5b5 (table test-table [id=0748ffc6b2394afbb7e1600e8849fdb3]) successfully deleted
I20260810 00:51:38.945180 14362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:38.907277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45078 (local address 127.10.97.65:42495)
0810 00:51:38.931880 (+ 24603us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:38.931886 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:38.931910 (+ 24us) server_negotiation.cc:407] Connection header received
0810 00:51:38.931963 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:38.931968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:38.932041 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:38.932157 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:38.934028 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:38.934865 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:38.936121 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:38.936534 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:38.936665 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:38.940498 (+ 3833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:38.940512 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:38.940528 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:38.940561 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:38.942988 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:38.943569 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:38.943576 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:38.943578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:38.943712 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:38.944049 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:38.944052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:38.944054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:38.944390 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:38.944607 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:38.944768 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 00:51:38.944921 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24453,"thread_start_us":24996,"threads_started":1}
W20260810 00:51:38.945745 13653 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45078
W20260810 00:51:38.946256 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 13463: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:51:38.946434 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 13463
I20260810 00:51:38.987495 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 13664
Aug 10 00:51:38 dist-test-slave-jlzn krb5kdc[13289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323096, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.97.66@KRBTEST.COM
I20260810 00:51:39.002050 14363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:38.988556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51518 (local address 127.10.97.66:35733)
0810 00:51:38.989022 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:38.989026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:38.989042 (+ 16us) server_negotiation.cc:407] Connection header received
0810 00:51:38.989095 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:38.989098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:38.989170 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:38.989264 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:38.990673 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:38.991505 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:38.992830 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:38.993216 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:38.993327 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:38.997119 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:38.997132 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:38.997156 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:38.997189 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:38.999756 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:39.000447 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:39.000454 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:39.000456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:39.000589 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:39.000860 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:39.000863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:39.000865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:39.001170 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:39.001390 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:39.001565 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 00:51:39.001720 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":227,"threads_started":1}
W20260810 00:51:39.002658 13861 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51518
W20260810 00:51:39.003198 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 13664: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:51:39.003320 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 13664
I20260810 00:51:39.033591 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 13872
Aug 10 00:51:39 dist-test-slave-jlzn krb5kdc[13289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323096, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.97.67@KRBTEST.COM
I20260810 00:51:39.046729 14364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:39.034212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59238 (local address 127.10.97.67:39951)
0810 00:51:39.034705 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:39.034709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:39.034724 (+ 15us) server_negotiation.cc:407] Connection header received
0810 00:51:39.034773 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:39.034776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:39.034830 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:39.034911 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:39.036034 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:39.036780 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:39.037849 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:39.038236 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:39.038316 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:39.042102 (+ 3786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:39.042115 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:39.042131 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:39.042162 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:39.044679 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:39.045219 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:39.045225 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:39.045227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:39.045341 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:39.045601 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:39.045604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:39.045606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:39.045927 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:39.046141 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:39.046315 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 00:51:39.046473 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":236,"threads_started":1}
W20260810 00:51:39.047354 14086 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59238
W20260810 00:51:39.047884 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 13872: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:51:39.048020 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 13872
I20260810 00:51:39.076952 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 13307
I20260810 00:51:39.077904 13414 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:39.188355 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 13307
2026-08-10T00:51:39Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8409 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:51:39 dist-test-slave-jlzn krb5kdc[14370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:51:39 dist-test-slave-jlzn krb5kdc[14370](info): set up 2 sockets
Aug 10 00:51:39 dist-test-slave-jlzn krb5kdc[14370](info): commencing operation
krb5kdc: starting...
W20260810 00:51:40.358428 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:51:40 dist-test-slave-jlzn krb5kdc[14370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:51:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:51:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:51:40.532564 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:44939
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:33641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:44939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:40.833544 14387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:40.833822 14387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:40.833951 14387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:40.840764 14387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:51:40.840868 14387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:40.840941 14387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:40.841018 14387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:51:40.841077 14387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:51:40.850135 14387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:44939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:44939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:40.851202 14387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:40.852571 14387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:40.863747 14394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:40.863775 14395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:40.864454 14398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:40.864923 14387 server_base.cc:1034] running on GCE node
I20260810 00:51:40.865563 14387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:40.866804 14387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:40.868081 14387 hybrid_clock.cc:648] HybridClock initialized: now 1786323100867973 us; error 88 us; skew 500 ppm
I20260810 00:51:40.868474 14387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:40 dist-test-slave-jlzn krb5kdc[14370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323100, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:40.876850 14387 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:51:40.878191 14387 webserver.cc:460] Webserver started at http://127.10.97.126:41233/ using document root <none> and password file <none>
I20260810 00:51:40.878814 14387 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:40.878926 14387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:40.879206 14387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:40.881961 14387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "4ef7305487bf47498ae01655a2c3871e"
format_stamp: "Formatted at 2026-08-10 00:51:40 on dist-test-slave-jlzn"
server_key: "8c91ca10bb3bba7cc273888d42a7d4ba"
server_key_iv: "ce1f10fa560a2b764f1ae6e33f4eb052"
server_key_version: "encryptionkey@0"
I20260810 00:51:40.882651 14387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "4ef7305487bf47498ae01655a2c3871e"
format_stamp: "Formatted at 2026-08-10 00:51:40 on dist-test-slave-jlzn"
server_key: "8c91ca10bb3bba7cc273888d42a7d4ba"
server_key_iv: "ce1f10fa560a2b764f1ae6e33f4eb052"
server_key_version: "encryptionkey@0"
I20260810 00:51:40.887359 14387 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260810 00:51:40.890534 14410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:40.891728 14387 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:51:40.891918 14387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "4ef7305487bf47498ae01655a2c3871e"
format_stamp: "Formatted at 2026-08-10 00:51:40 on dist-test-slave-jlzn"
server_key: "8c91ca10bb3bba7cc273888d42a7d4ba"
server_key_iv: "ce1f10fa560a2b764f1ae6e33f4eb052"
server_key_version: "encryptionkey@0"
I20260810 00:51:40.892088 14387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:40.911294 14387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:40.914789 14387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:40.915030 14387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:40.931438 14387 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:44939
I20260810 00:51:40.931468 14492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:44939 every 8 connection(s)
I20260810 00:51:40.932650 14387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:51:40.936472 14493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:40.937829 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 14387
I20260810 00:51:40.938038 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:51:40.938643 10629 external_mini_cluster.cc:1428] Setting key a6bbe03a91119056e859a2a7688dfe90
I20260810 00:51:40.949611 14493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e: Bootstrap starting.
Aug 10 00:51:40 dist-test-slave-jlzn krb5kdc[14370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:40.952827 14493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:40.954090 14493 log.cc:824] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:40.957167 14493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e: No bootstrap required, opened a new log
I20260810 00:51:40.961136 14502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:40.940409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60064 (local address 127.10.97.126:44939)
0810 00:51:40.941221 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:40.941236 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:40.941293 (+ 57us) server_negotiation.cc:407] Connection header received
0810 00:51:40.942919 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:40.942950 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:40.943407 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:40.943822 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:40.945049 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:40.947021 (+ 1972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:40.948235 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:40.949182 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:40.949366 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:40.953187 (+ 3821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:40.953199 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:40.953222 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:40.953276 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:40.956942 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:40.957605 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:40.957614 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:40.957621 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:40.957829 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:40.958147 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:40.958151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:40.958155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:40.958829 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:40.959137 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:40.959448 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 00:51:40.959879 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":346,"threads_started":1}
I20260810 00:51:40.965728 14493 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ef7305487bf47498ae01655a2c3871e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44939 } }
I20260810 00:51:40.966085 14493 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:40.966192 14493 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef7305487bf47498ae01655a2c3871e, State: Initialized, Role: FOLLOWER
I20260810 00:51:40.966742 14493 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ef7305487bf47498ae01655a2c3871e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44939 } }
I20260810 00:51:40.967010 14493 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:51:40.967140 14493 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:51:40.967291 14493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:40.969316 14493 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ef7305487bf47498ae01655a2c3871e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44939 } }
I20260810 00:51:40.969762 14493 leader_election.cc:302] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ef7305487bf47498ae01655a2c3871e; no voters:
I20260810 00:51:40.970257 14493 leader_election.cc:288] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:51:40.970338 14505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:40.970943 14505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e [term 1 LEADER]: Becoming Leader. State: Replica: 4ef7305487bf47498ae01655a2c3871e, State: Running, Role: LEADER
I20260810 00:51:40.971406 14505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ef7305487bf47498ae01655a2c3871e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44939 } }
I20260810 00:51:40.972118 14493 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:51:40.973876 14507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ef7305487bf47498ae01655a2c3871e. Latest consensus state: current_term: 1 leader_uuid: "4ef7305487bf47498ae01655a2c3871e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ef7305487bf47498ae01655a2c3871e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44939 } } }
I20260810 00:51:40.974220 14507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e [sys.catalog]: This master's current role is: LEADER
I20260810 00:51:40.974263 14506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ef7305487bf47498ae01655a2c3871e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ef7305487bf47498ae01655a2c3871e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44939 } } }
I20260810 00:51:40.974660 14506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e [sys.catalog]: This master's current role is: LEADER
W20260810 00:51:40.982841 14528 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 00:51:40.982935 14528 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 00:51:40.983047 14510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:51:40.985148 14510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:51:40.993014 14510 catalog_manager.cc:1347] Generated new cluster ID: 9bd50c2647734a96acd74c7481fc3a71
I20260810 00:51:40.993121 14510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:51:41.005530 14510 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:51:41.006647 14510 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:51:41.017333 14510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ef7305487bf47498ae01655a2c3871e: Generated new TSK 0
I20260810 00:51:41.018132 14510 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
I20260810 00:51:41.071028 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:44939
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:33641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:41.378433 14537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:41.378660 14537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:41.378780 14537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:41.385542 14537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:41.385636 14537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:41.385775 14537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:51:41.394755 14537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:44939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:41.395875 14537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:41.397003 14537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:41.408726 14545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:41.408831 14547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:41.408954 14549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:41.409621 14537 server_base.cc:1034] running on GCE node
I20260810 00:51:41.410167 14537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:41.410959 14537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:41.412155 14537 hybrid_clock.cc:648] HybridClock initialized: now 1786323101412096 us; error 44 us; skew 500 ppm
I20260810 00:51:41.412424 14537 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:41 dist-test-slave-jlzn krb5kdc[14370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323101, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:41.415228 14537 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:51:41.416501 14537 webserver.cc:460] Webserver started at http://127.10.97.65:39355/ using document root <none> and password file <none>
I20260810 00:51:41.417083 14537 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:41.417204 14537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:41.417472 14537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:41.420171 14537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "017ed1b920b84c05960344a7c437e3d1"
format_stamp: "Formatted at 2026-08-10 00:51:41 on dist-test-slave-jlzn"
server_key: "fcb325c563b64c46d0cf205c38f864f9"
server_key_iv: "117a47e203197156025b38394c5d81f6"
server_key_version: "encryptionkey@0"
I20260810 00:51:41.420866 14537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "017ed1b920b84c05960344a7c437e3d1"
format_stamp: "Formatted at 2026-08-10 00:51:41 on dist-test-slave-jlzn"
server_key: "fcb325c563b64c46d0cf205c38f864f9"
server_key_iv: "117a47e203197156025b38394c5d81f6"
server_key_version: "encryptionkey@0"
I20260810 00:51:41.425427 14537 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:51:41.428483 14559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:41.429603 14537 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:51:41.429742 14537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "017ed1b920b84c05960344a7c437e3d1"
format_stamp: "Formatted at 2026-08-10 00:51:41 on dist-test-slave-jlzn"
server_key: "fcb325c563b64c46d0cf205c38f864f9"
server_key_iv: "117a47e203197156025b38394c5d81f6"
server_key_version: "encryptionkey@0"
I20260810 00:51:41.429908 14537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:41.448518 14537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:41.451851 14537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:41.452111 14537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:41.452919 14537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:41.454075 14537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:41.454176 14537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:41.454294 14537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:41.454348 14537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:41.482852 14537 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:35085
I20260810 00:51:41.482908 14751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:35085 every 8 connection(s)
I20260810 00:51:41.483976 14537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:51:41.486641 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 14537
I20260810 00:51:41.486825 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:51:41.487615 10629 external_mini_cluster.cc:1428] Setting key d6990fef499c666cfae50a7612d24ed3
Aug 10 00:51:41 dist-test-slave-jlzn krb5kdc[14370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323101, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:41.504091 14755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:41.486125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:51225 (local address 127.10.97.126:44939)
0810 00:51:41.486531 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:41.486537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:41.487625 (+ 1088us) server_negotiation.cc:407] Connection header received
0810 00:51:41.489702 (+ 2077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:41.489705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:41.489773 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:41.489860 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:41.491648 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:41.492370 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:41.494316 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:41.494711 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:41.494811 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:41.498333 (+ 3522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:41.498340 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:41.498344 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:41.498385 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:41.501370 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:41.501983 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:41.501988 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:41.501992 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:41.502144 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:41.502509 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:41.502514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:41.502517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:41.502848 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:41.502983 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:41.503514 (+ 531us) server_negotiation.cc:299] Negotiation successful
0810 00:51:41.503707 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":195,"threads_started":1}
I20260810 00:51:41.504667 14757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:41.486651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:44939 (local address 127.10.97.65:51225)
0810 00:51:41.487463 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:41.487508 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:41.489455 (+ 1947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:41.490114 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:41.490125 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:41.490753 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:41.491468 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:41.491507 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:41.492512 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:41.492516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:41.494201 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:41.494215 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:41.494929 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:41.494936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:41.495200 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:41.495857 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:41.495891 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:41.498127 (+ 2236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:41.501489 (+ 3362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:41.501496 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:41.501505 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:41.501872 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:41.502237 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:41.502240 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:41.502243 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:41.502399 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:41.502944 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:41.502949 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:41.503208 (+ 259us) client_negotiation.cc:769] Sending connection context
0810 00:51:41.503406 (+ 198us) client_negotiation.cc:240] Negotiation successful
0810 00:51:41.503716 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":313,"threads_started":1}
I20260810 00:51:41.505928 14753 heartbeater.cc:347] Connected to a master server at 127.10.97.126:44939
I20260810 00:51:41.506224 14753 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:41.507011 14753 heartbeater.cc:511] Master 127.10.97.126:44939 requested a full tablet report, sending...
I20260810 00:51:41.509061 14440 ts_manager.cc:194] Registered new tserver with Master: 017ed1b920b84c05960344a7c437e3d1 (127.10.97.65:35085)
WARNING: no policy specified for kudu/127.10.97.66@KRBTEST.COM; defaulting to no policy
I20260810 00:51:41.510469 14440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:51225
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
I20260810 00:51:41.532558 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:44939
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:33641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:41.834560 14762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:41.834824 14762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:41.834964 14762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:41.841872 14762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:41.841981 14762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:41.842144 14762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:51:41.851197 14762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:44939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:41.852305 14762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:41.853482 14762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:41.866030 14770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:41.866074 14771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:41.866096 14778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:41.867159 14762 server_base.cc:1034] running on GCE node
I20260810 00:51:41.867671 14762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:41.868500 14762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:41.869699 14762 hybrid_clock.cc:648] HybridClock initialized: now 1786323101869660 us; error 29 us; skew 500 ppm
I20260810 00:51:41.869975 14762 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:41 dist-test-slave-jlzn krb5kdc[14370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323101, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:41.872895 14762 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:51:41.874208 14762 webserver.cc:460] Webserver started at http://127.10.97.66:38143/ using document root <none> and password file <none>
I20260810 00:51:41.874814 14762 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:41.874926 14762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:41.875219 14762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:41.877888 14762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "6c917827c736487d8e114fb158de0d88"
format_stamp: "Formatted at 2026-08-10 00:51:41 on dist-test-slave-jlzn"
server_key: "3af485b0391e4e16d53ca660788460e1"
server_key_iv: "d35e7b3654cfa113bf93992c34b859aa"
server_key_version: "encryptionkey@0"
I20260810 00:51:41.878626 14762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "6c917827c736487d8e114fb158de0d88"
format_stamp: "Formatted at 2026-08-10 00:51:41 on dist-test-slave-jlzn"
server_key: "3af485b0391e4e16d53ca660788460e1"
server_key_iv: "d35e7b3654cfa113bf93992c34b859aa"
server_key_version: "encryptionkey@0"
I20260810 00:51:41.883334 14762 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260810 00:51:41.886528 14793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:41.887662 14762 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 00:51:41.887848 14762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "6c917827c736487d8e114fb158de0d88"
format_stamp: "Formatted at 2026-08-10 00:51:41 on dist-test-slave-jlzn"
server_key: "3af485b0391e4e16d53ca660788460e1"
server_key_iv: "d35e7b3654cfa113bf93992c34b859aa"
server_key_version: "encryptionkey@0"
I20260810 00:51:41.888034 14762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:41.903628 14762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:41.907079 14762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:41.907325 14762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:41.908190 14762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:41.909418 14762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:41.909495 14762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:41.909607 14762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:41.909660 14762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:41.940474 14762 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:45817
I20260810 00:51:41.940519 14971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:45817 every 8 connection(s)
I20260810 00:51:41.941619 14762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:51:41.946697 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 14762
I20260810 00:51:41.946875 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:51:41.947276 10629 external_mini_cluster.cc:1428] Setting key 10deaf9a1334643cff168c4a52ae4acb
Aug 10 00:51:41 dist-test-slave-jlzn krb5kdc[14370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323101, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:41.961290 14755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:41.944028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:56985 (local address 127.10.97.126:44939)
0810 00:51:41.944226 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:41.944232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:41.945376 (+ 1144us) server_negotiation.cc:407] Connection header received
0810 00:51:41.947770 (+ 2394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:41.947775 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:41.947878 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:41.947964 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:41.949609 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:41.950301 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:41.951786 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:41.952212 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:41.952321 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:41.955443 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:41.955448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:41.955451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:41.955480 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:41.958460 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:41.959105 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:41.959111 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:41.959115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:41.959267 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:41.959651 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:41.959656 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:41.959659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:41.960016 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:41.960161 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:41.960755 (+ 594us) server_negotiation.cc:299] Negotiation successful
0810 00:51:41.960977 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 00:51:41.961915 14977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:41.944448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:44939 (local address 127.10.97.66:56985)
0810 00:51:41.945203 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:41.945255 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:41.947523 (+ 2268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:41.948240 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:41.948251 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:41.948922 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:41.949455 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:41.949471 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:41.950446 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:41.950450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:41.951651 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:41.951663 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:41.952426 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:41.952434 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:41.952638 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:41.953245 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:41.953266 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:41.955250 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:41.958589 (+ 3339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:41.958595 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:41.958605 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:41.958987 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:41.959360 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:41.959363 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:41.959365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:41.959543 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:41.960126 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:41.960131 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:41.960400 (+ 269us) client_negotiation.cc:769] Sending connection context
0810 00:51:41.960629 (+ 229us) client_negotiation.cc:240] Negotiation successful
0810 00:51:41.960950 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":277,"threads_started":1}
I20260810 00:51:41.963212 14973 heartbeater.cc:347] Connected to a master server at 127.10.97.126:44939
I20260810 00:51:41.963469 14973 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:41.964165 14973 heartbeater.cc:511] Master 127.10.97.126:44939 requested a full tablet report, sending...
I20260810 00:51:41.965521 14440 ts_manager.cc:194] Registered new tserver with Master: 6c917827c736487d8e114fb158de0d88 (127.10.97.66:45817)
I20260810 00:51:41.966290 14440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:56985
WARNING: no policy specified for kudu/127.10.97.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
I20260810 00:51:41.990552 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:44939
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:33641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:42.292016 14980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:42.292281 14980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:42.292414 14980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:42.299522 14980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:42.299641 14980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:42.299793 14980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:51:42.308883 14980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:44939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:42.309995 14980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:42.311192 14980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:42.322978 14989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:42.323019 14994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:42.323082 14996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:42.323724 14980 server_base.cc:1034] running on GCE node
I20260810 00:51:42.324321 14980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:42.325157 14980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:42.326354 14980 hybrid_clock.cc:648] HybridClock initialized: now 1786323102326309 us; error 31 us; skew 500 ppm
I20260810 00:51:42.326644 14980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:42 dist-test-slave-jlzn krb5kdc[14370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323102, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:42.329509 14980 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:51:42.330829 14980 webserver.cc:460] Webserver started at http://127.10.97.67:44689/ using document root <none> and password file <none>
I20260810 00:51:42.331424 14980 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:42.331552 14980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:42.331875 14980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:42.334641 14980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "a660f741daeb483dbc621b835e40fbc7"
format_stamp: "Formatted at 2026-08-10 00:51:42 on dist-test-slave-jlzn"
server_key: "99b21e5d819e94a73e5c121e80fa95b1"
server_key_iv: "ab29161bb5cdec51f8942c4c0c7607b8"
server_key_version: "encryptionkey@0"
I20260810 00:51:42.335376 14980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "a660f741daeb483dbc621b835e40fbc7"
format_stamp: "Formatted at 2026-08-10 00:51:42 on dist-test-slave-jlzn"
server_key: "99b21e5d819e94a73e5c121e80fa95b1"
server_key_iv: "ab29161bb5cdec51f8942c4c0c7607b8"
server_key_version: "encryptionkey@0"
I20260810 00:51:42.340018 14980 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260810 00:51:42.343056 15010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:42.344265 14980 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:51:42.344409 14980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "a660f741daeb483dbc621b835e40fbc7"
format_stamp: "Formatted at 2026-08-10 00:51:42 on dist-test-slave-jlzn"
server_key: "99b21e5d819e94a73e5c121e80fa95b1"
server_key_iv: "ab29161bb5cdec51f8942c4c0c7607b8"
server_key_version: "encryptionkey@0"
I20260810 00:51:42.344574 14980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:42.370997 14980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:42.374612 14980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:42.374862 14980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:42.375690 14980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:42.376929 14980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:42.377009 14980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:42.377121 14980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:42.377174 14980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:42.406219 14980 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:36851
I20260810 00:51:42.406275 15191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:36851 every 8 connection(s)
I20260810 00:51:42.407366 14980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:51:42.414326 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 14980
I20260810 00:51:42.414501 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:51:42.414913 10629 external_mini_cluster.cc:1428] Setting key b3983477abb4be8d14763834aad0bf9b
Aug 10 00:51:42 dist-test-slave-jlzn krb5kdc[14370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323102, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:42.425746 14755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:42.409612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:42287 (local address 127.10.97.126:44939)
0810 00:51:42.409769 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:42.409774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:42.410832 (+ 1058us) server_negotiation.cc:407] Connection header received
0810 00:51:42.412582 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:42.412586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:42.412661 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:42.412743 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:42.414047 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:42.414826 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:42.416848 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:42.417435 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:42.417557 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:42.420718 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:42.420723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:42.420726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:42.420757 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:42.422998 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:42.423600 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:42.423603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:42.423605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:42.423719 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:42.424140 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:42.424146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:42.424148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:42.424418 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:42.424552 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:42.425180 (+ 628us) server_negotiation.cc:299] Negotiation successful
0810 00:51:42.425415 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 00:51:42.426504 15195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:42.410028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:44939 (local address 127.10.97.67:42287)
0810 00:51:42.410672 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:42.410708 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:42.412356 (+ 1648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:42.412922 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:42.412930 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:42.413406 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:42.413922 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:42.413938 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:42.414984 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:42.414989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:42.416706 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:42.416728 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:42.417627 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:42.417636 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:42.417908 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:42.418515 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:42.418536 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:42.420538 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:42.423117 (+ 2579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:42.423123 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:42.423141 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:42.423513 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:42.423789 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:42.423792 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:42.423794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:42.424026 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:42.424539 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:42.424546 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:42.424825 (+ 279us) client_negotiation.cc:769] Sending connection context
0810 00:51:42.425049 (+ 224us) client_negotiation.cc:240] Negotiation successful
0810 00:51:42.425500 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":254,"threads_started":1}
I20260810 00:51:42.427642 15192 heartbeater.cc:347] Connected to a master server at 127.10.97.126:44939
I20260810 00:51:42.427924 15192 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:42.428594 15192 heartbeater.cc:511] Master 127.10.97.126:44939 requested a full tablet report, sending...
I20260810 00:51:42.429935 14440 ts_manager.cc:194] Registered new tserver with Master: a660f741daeb483dbc621b835e40fbc7 (127.10.97.67:36851)
I20260810 00:51:42.430698 14440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:42287
I20260810 00:51:42.441573 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 00:51:42.476385 14755 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:51:42.474056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35784 (local address 127.10.97.126:44939)
0810 00:51:42.474227 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:42.474231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:42.474707 (+ 476us) server_negotiation.cc:407] Connection header received
0810 00:51:42.474870 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:42.474874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:42.474937 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:42.475028 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:42.476189 (+ 1161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35784: BlockingRecv error: recv got EOF from 127.0.0.1:35784 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 00:51:42.479318 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 14537
W20260810 00:51:42.481796 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 14537: Not authorized: Client connection negotiation failed: client connection to 127.10.97.65:35085: server requires authentication, but client does not have Kerberos credentials available
I20260810 00:51:42.481946 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 14537
I20260810 00:51:42.508159 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 14762
W20260810 00:51:42.510855 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 14762: Not authorized: Client connection negotiation failed: client connection to 127.10.97.66:45817: server requires authentication, but client does not have Kerberos credentials available
I20260810 00:51:42.511036 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 14762
I20260810 00:51:42.534646 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 14980
W20260810 00:51:42.537513 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 14980: Not authorized: Client connection negotiation failed: client connection to 127.10.97.67:36851: server requires authentication, but client does not have Kerberos credentials available
I20260810 00:51:42.537643 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 14980
I20260810 00:51:42.557703 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 14387
I20260810 00:51:42.558784 14484 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:42.660003 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 14387
2026-08-10T00:51:42Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3468 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:51:42 dist-test-slave-jlzn krb5kdc[15220](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:51:42 dist-test-slave-jlzn krb5kdc[15220](info): set up 2 sockets
Aug 10 00:51:42 dist-test-slave-jlzn krb5kdc[15220](info): commencing operation
krb5kdc: starting...
W20260810 00:51:43.856439 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260810 00:51:43.989269 10629 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.133s user 0.002s sys 0.006s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260810 00:51:44.155049 10629 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.166s user 0.001s sys 0.009s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260810 00:51:44.317749 10629 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.163s user 0.000s sys 0.007s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/krb5kdc/test-user.keytab.
W20260810 00:51:44.712793 10629 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.395s user 0.000s sys 0.007s
Aug 10 00:51:44 dist-test-slave-jlzn krb5kdc[15220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:51:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:51:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
W20260810 00:51:45.158296 10629 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.10.97.126: real 0.394s user 0.001s sys 0.010s
I20260810 00:51:45.159284 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:44133
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:36223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:44133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:45.468832 15236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:45.469069 15236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:45.469198 15236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:45.475956 15236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:51:45.476049 15236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:45.476125 15236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:45.476202 15236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:51:45.476266 15236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:51:45.485263 15236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:36223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:44133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:44133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:45.486307 15236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:45.487489 15236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:45.498018 15242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:45.498106 15243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:45.498131 15245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:45.499006 15236 server_base.cc:1034] running on GCE node
I20260810 00:51:45.499703 15236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:45.500939 15236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:45.502138 15236 hybrid_clock.cc:648] HybridClock initialized: now 1786323105502089 us; error 31 us; skew 500 ppm
I20260810 00:51:45.502460 15236 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:45 dist-test-slave-jlzn krb5kdc[15220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323105, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:45.505370 15236 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:51:45.506731 15236 webserver.cc:460] Webserver started at http://127.10.97.126:44369/ using document root <none> and password file <none>
I20260810 00:51:45.507304 15236 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:45.507404 15236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:45.507660 15236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:45.510381 15236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "21d42748ca58426ab23ca3a780b140c2"
format_stamp: "Formatted at 2026-08-10 00:51:45 on dist-test-slave-jlzn"
server_key: "dc02a49d7bdda2bc9b7526e67dc501b8"
server_key_iv: "561c15b3e45aec0f3239eba7390d0db5"
server_key_version: "encryptionkey@0"
I20260810 00:51:45.511088 15236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "21d42748ca58426ab23ca3a780b140c2"
format_stamp: "Formatted at 2026-08-10 00:51:45 on dist-test-slave-jlzn"
server_key: "dc02a49d7bdda2bc9b7526e67dc501b8"
server_key_iv: "561c15b3e45aec0f3239eba7390d0db5"
server_key_version: "encryptionkey@0"
I20260810 00:51:45.515734 15236 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260810 00:51:45.518836 15257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:45.520072 15236 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260810 00:51:45.520229 15236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "21d42748ca58426ab23ca3a780b140c2"
format_stamp: "Formatted at 2026-08-10 00:51:45 on dist-test-slave-jlzn"
server_key: "dc02a49d7bdda2bc9b7526e67dc501b8"
server_key_iv: "561c15b3e45aec0f3239eba7390d0db5"
server_key_version: "encryptionkey@0"
I20260810 00:51:45.520395 15236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:45.534468 15236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:45.537942 15236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:45.538161 15236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:45.553045 15236 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:44133
I20260810 00:51:45.553087 15342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:44133 every 8 connection(s)
I20260810 00:51:45.554212 15236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:51:45.556618 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 15236
I20260810 00:51:45.556841 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:51:45.557427 10629 external_mini_cluster.cc:1428] Setting key f6288eb751f78896b15f0ccc57ef2b92
I20260810 00:51:45.557578 15344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:45.571229 15344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2: Bootstrap starting.
Aug 10 00:51:45 dist-test-slave-jlzn krb5kdc[15220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:45.574858 15344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:45.576159 15344 log.cc:824] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:45.578724 15344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2: No bootstrap required, opened a new log
I20260810 00:51:45.579843 15348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:45.559017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33620 (local address 127.10.97.126:44133)
0810 00:51:45.559732 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:45.559746 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:45.559794 (+ 48us) server_negotiation.cc:407] Connection header received
0810 00:51:45.562163 (+ 2369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:45.562193 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:45.562651 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:45.563082 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:45.564428 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:45.566353 (+ 1925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:45.567519 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:45.568589 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:45.568764 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:45.572511 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:45.572523 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:45.572541 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:45.572590 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:45.576183 (+ 3593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:45.576808 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:45.576814 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:45.576822 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:45.576987 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:45.577345 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:45.577349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:45.577351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:45.577842 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:45.578095 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:45.578535 (+ 440us) server_negotiation.cc:299] Negotiation successful
0810 00:51:45.578919 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":301,"threads_started":1}
I20260810 00:51:45.583233 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:51:45.586001 15344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21d42748ca58426ab23ca3a780b140c2" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44133 } }
I20260810 00:51:45.586313 15344 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:45.586403 15344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21d42748ca58426ab23ca3a780b140c2, State: Initialized, Role: FOLLOWER
I20260810 00:51:45.586894 15344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21d42748ca58426ab23ca3a780b140c2" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44133 } }
I20260810 00:51:45.587105 15344 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:51:45.587227 15344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:51:45.587371 15344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:45.589366 15344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21d42748ca58426ab23ca3a780b140c2" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44133 } }
I20260810 00:51:45.589802 15344 leader_election.cc:302] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21d42748ca58426ab23ca3a780b140c2; no voters:
I20260810 00:51:45.590384 15344 leader_election.cc:288] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:51:45.590476 15350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:45.591568 15350 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2 [term 1 LEADER]: Becoming Leader. State: Replica: 21d42748ca58426ab23ca3a780b140c2, State: Running, Role: LEADER
I20260810 00:51:45.592165 15350 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21d42748ca58426ab23ca3a780b140c2" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44133 } }
I20260810 00:51:45.592555 15344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:51:45.594735 15352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21d42748ca58426ab23ca3a780b140c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21d42748ca58426ab23ca3a780b140c2" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44133 } } }
I20260810 00:51:45.595111 15352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2 [sys.catalog]: This master's current role is: LEADER
I20260810 00:51:45.597038 15358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:51:45.597155 15353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21d42748ca58426ab23ca3a780b140c2. Latest consensus state: current_term: 1 leader_uuid: "21d42748ca58426ab23ca3a780b140c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21d42748ca58426ab23ca3a780b140c2" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44133 } } }
I20260810 00:51:45.597424 15353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2 [sys.catalog]: This master's current role is: LEADER
I20260810 00:51:45.602468 15358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:51:45.627681 15358 catalog_manager.cc:1347] Generated new cluster ID: 9c789ddb27304acfa929c5c4807b6cb7
I20260810 00:51:45.627784 15358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:51:45.636617 15348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:45.627465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33636 (local address 127.10.97.126:44133)
0810 00:51:45.627727 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:45.627733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:45.628403 (+ 670us) server_negotiation.cc:407] Connection header received
0810 00:51:45.628601 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:45.628611 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:45.628706 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:45.628814 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:45.629989 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:45.631074 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:45.632804 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:45.633497 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:45.633627 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:45.634717 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:45.634724 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:45.634729 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:51:45.635648 (+ 919us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:45.635711 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:51:45.635745 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:45.635907 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:45.636022 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 00:51:45.636115 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
I20260810 00:51:45.655097 15358 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:51:45.656327 15358 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:51:45.668269 15358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21d42748ca58426ab23ca3a780b140c2: Generated new TSK 0
I20260810 00:51:45.668970 15358 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:51:45.679322 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 15236
I20260810 00:51:45.680621 15338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:45.770027 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 15236
2026-08-10T00:51:45Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (3106 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:51:45 dist-test-slave-jlzn krb5kdc[15393](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:51:45 dist-test-slave-jlzn krb5kdc[15393](info): set up 2 sockets
Aug 10 00:51:45 dist-test-slave-jlzn krb5kdc[15393](info): commencing operation
krb5kdc: starting...
W20260810 00:51:46.948594 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:51:47 dist-test-slave-jlzn krb5kdc[15393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:51:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:51:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:51:47.126461 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:35885
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:35885
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:47.427386 15413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:47.427634 15413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:47.427726 15413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:47.434540 15413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:51:47.434633 15413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:47.434684 15413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:47.434738 15413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:51:47.434784 15413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:51:47.443774 15413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:35885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:35885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:47.444828 15413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:47.445935 15413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:47.456193 15418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:47.456228 15424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:47.456269 15422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:47.457244 15413 server_base.cc:1034] running on GCE node
I20260810 00:51:47.457882 15413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:47.459028 15413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:47.460217 15413 hybrid_clock.cc:648] HybridClock initialized: now 1786323107460180 us; error 27 us; skew 500 ppm
I20260810 00:51:47.460521 15413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:47 dist-test-slave-jlzn krb5kdc[15393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323107, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:47.467232 15413 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:51:47.468581 15413 webserver.cc:460] Webserver started at http://127.10.97.126:37549/ using document root <none> and password file <none>
I20260810 00:51:47.469144 15413 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:47.469246 15413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:47.469472 15413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:47.472186 15413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "71a586e26dc940c9b5fba110ea0733b2"
format_stamp: "Formatted at 2026-08-10 00:51:47 on dist-test-slave-jlzn"
server_key: "00af22faa72885bb904b80f0b66cdf9c"
server_key_iv: "1b510ae5508f7168885aa669708df095"
server_key_version: "encryptionkey@0"
I20260810 00:51:47.472853 15413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "71a586e26dc940c9b5fba110ea0733b2"
format_stamp: "Formatted at 2026-08-10 00:51:47 on dist-test-slave-jlzn"
server_key: "00af22faa72885bb904b80f0b66cdf9c"
server_key_iv: "1b510ae5508f7168885aa669708df095"
server_key_version: "encryptionkey@0"
I20260810 00:51:47.477427 15413 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:51:47.480525 15433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:47.481696 15413 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:51:47.481849 15413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "71a586e26dc940c9b5fba110ea0733b2"
format_stamp: "Formatted at 2026-08-10 00:51:47 on dist-test-slave-jlzn"
server_key: "00af22faa72885bb904b80f0b66cdf9c"
server_key_iv: "1b510ae5508f7168885aa669708df095"
server_key_version: "encryptionkey@0"
I20260810 00:51:47.482008 15413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:47.498147 15413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:47.501557 15413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:47.501801 15413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:47.517882 15413 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:35885
I20260810 00:51:47.517923 15519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:35885 every 8 connection(s)
I20260810 00:51:47.519006 15413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:51:47.519388 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 15413
I20260810 00:51:47.519593 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:51:47.520339 10629 external_mini_cluster.cc:1428] Setting key 2a8508d08d02af91ba61aada9c46f5b6
I20260810 00:51:47.522842 15521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 00:51:47 dist-test-slave-jlzn krb5kdc[15393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:47.532768 15521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2: Bootstrap starting.
I20260810 00:51:47.535778 15521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:47.536755 15521 log.cc:824] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:47.539232 15521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2: No bootstrap required, opened a new log
I20260810 00:51:47.540163 15527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:47.522076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43834 (local address 127.10.97.126:35885)
0810 00:51:47.522790 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:47.522801 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:47.522841 (+ 40us) server_negotiation.cc:407] Connection header received
0810 00:51:47.524417 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:47.524436 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:47.524750 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:47.525088 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:47.526430 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:47.527686 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:47.528886 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:47.529506 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:47.529688 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:47.533588 (+ 3900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:47.533595 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:47.533613 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:47.533659 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:47.536528 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:47.537190 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:47.537196 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:47.537201 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:47.537371 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:47.537725 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:47.537729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:47.537731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:47.538185 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:47.538475 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:47.538873 (+ 398us) server_negotiation.cc:299] Negotiation successful
0810 00:51:47.539230 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":320,"threads_started":1}
I20260810 00:51:47.547322 15521 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71a586e26dc940c9b5fba110ea0733b2" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35885 } }
I20260810 00:51:47.547653 15521 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:47.547745 15521 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71a586e26dc940c9b5fba110ea0733b2, State: Initialized, Role: FOLLOWER
I20260810 00:51:47.548296 15521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71a586e26dc940c9b5fba110ea0733b2" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35885 } }
I20260810 00:51:47.548511 15521 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:51:47.548645 15521 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:51:47.548804 15521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:47.551112 15521 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71a586e26dc940c9b5fba110ea0733b2" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35885 } }
I20260810 00:51:47.551586 15521 leader_election.cc:302] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71a586e26dc940c9b5fba110ea0733b2; no voters:
I20260810 00:51:47.552160 15521 leader_election.cc:288] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:51:47.552248 15529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:47.552852 15529 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2 [term 1 LEADER]: Becoming Leader. State: Replica: 71a586e26dc940c9b5fba110ea0733b2, State: Running, Role: LEADER
I20260810 00:51:47.553313 15529 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71a586e26dc940c9b5fba110ea0733b2" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35885 } }
I20260810 00:51:47.553966 15521 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:51:47.555683 15530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71a586e26dc940c9b5fba110ea0733b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71a586e26dc940c9b5fba110ea0733b2" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35885 } } }
I20260810 00:51:47.556030 15530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2 [sys.catalog]: This master's current role is: LEADER
I20260810 00:51:47.556705 15533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71a586e26dc940c9b5fba110ea0733b2. Latest consensus state: current_term: 1 leader_uuid: "71a586e26dc940c9b5fba110ea0733b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71a586e26dc940c9b5fba110ea0733b2" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35885 } } }
I20260810 00:51:47.557021 15533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2 [sys.catalog]: This master's current role is: LEADER
I20260810 00:51:47.557600 15537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:51:47.562485 15537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:51:47.570642 15537 catalog_manager.cc:1347] Generated new cluster ID: 6df125d325284b879758834a65de57c7
I20260810 00:51:47.570741 15537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:51:47.590108 15537 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:51:47.591233 15537 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:51:47.597728 15537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71a586e26dc940c9b5fba110ea0733b2: Generated new TSK 0
I20260810 00:51:47.598464 15537 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
I20260810 00:51:47.653493 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:35885
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:47.962986 15573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:47.963241 15573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:47.963364 15573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:47.970242 15573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:47.970351 15573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:47.970495 15573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:51:47.979492 15573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:35885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:47.980669 15573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:47.981845 15573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:47.993749 15583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:47.993822 15584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:47.993785 15586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:47.994729 15573 server_base.cc:1034] running on GCE node
I20260810 00:51:47.995255 15573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:47.996107 15573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:47.997298 15573 hybrid_clock.cc:648] HybridClock initialized: now 1786323107997259 us; error 31 us; skew 500 ppm
I20260810 00:51:47.997581 15573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:47 dist-test-slave-jlzn krb5kdc[15393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323107, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:48.000661 15573 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:51:48.002357 15573 webserver.cc:460] Webserver started at http://127.10.97.65:38759/ using document root <none> and password file <none>
I20260810 00:51:48.002970 15573 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:48.003086 15573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:48.003360 15573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:48.006150 15573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "2e2adacd8c40462f87fa32ddddb07942"
format_stamp: "Formatted at 2026-08-10 00:51:47 on dist-test-slave-jlzn"
server_key: "836b5900b9fde00fe931348291288b42"
server_key_iv: "646cb1ffdd7c593cce207dc46d59f767"
server_key_version: "encryptionkey@0"
I20260810 00:51:48.006879 15573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "2e2adacd8c40462f87fa32ddddb07942"
format_stamp: "Formatted at 2026-08-10 00:51:47 on dist-test-slave-jlzn"
server_key: "836b5900b9fde00fe931348291288b42"
server_key_iv: "646cb1ffdd7c593cce207dc46d59f767"
server_key_version: "encryptionkey@0"
I20260810 00:51:48.011603 15573 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.006s
I20260810 00:51:48.014842 15599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:48.016106 15573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 00:51:48.016256 15573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "2e2adacd8c40462f87fa32ddddb07942"
format_stamp: "Formatted at 2026-08-10 00:51:47 on dist-test-slave-jlzn"
server_key: "836b5900b9fde00fe931348291288b42"
server_key_iv: "646cb1ffdd7c593cce207dc46d59f767"
server_key_version: "encryptionkey@0"
I20260810 00:51:48.016434 15573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:48.036278 15573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:48.040037 15573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:48.040282 15573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:48.041124 15573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:48.042371 15573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:48.042452 15573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:48.042572 15573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:48.042629 15573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:48.072217 15573 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:41495
I20260810 00:51:48.072289 15779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:41495 every 8 connection(s)
I20260810 00:51:48.073345 15573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:51:48.079874 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 15573
I20260810 00:51:48.080111 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:51:48.080817 10629 external_mini_cluster.cc:1428] Setting key a941732a93d7ca25c31b1ea8bb02a168
Aug 10 00:51:48 dist-test-slave-jlzn krb5kdc[15393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323107, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:48.092525 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:48.075562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:57441 (local address 127.10.97.126:35885)
0810 00:51:48.076000 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:48.076005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:48.076878 (+ 873us) server_negotiation.cc:407] Connection header received
0810 00:51:48.078578 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:48.078584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:48.078663 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:48.078762 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:48.080445 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:48.081229 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:48.083051 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:48.083436 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:48.083536 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:48.087333 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:48.087338 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:48.087341 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:48.087373 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:48.089740 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:48.090484 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:48.090488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:48.090490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:48.090598 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:48.090976 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:48.090981 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:48.090984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:48.091343 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:48.091483 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:48.092099 (+ 616us) server_negotiation.cc:299] Negotiation successful
0810 00:51:48.092275 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":230,"threads_started":1}
I20260810 00:51:48.093564 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:48.076077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:35885 (local address 127.10.97.65:57441)
0810 00:51:48.076731 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:48.076769 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:48.078327 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:48.078989 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:48.078996 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:48.079489 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:48.080263 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:48.080292 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:48.081384 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:48.081389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:48.082914 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:48.082930 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:48.083656 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:48.083662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:48.083943 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:48.084566 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:48.084599 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:48.087125 (+ 2526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:48.089871 (+ 2746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:48.089879 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:48.089891 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:48.090385 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:48.090689 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:48.090694 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:48.090697 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:48.090871 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:48.091443 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:48.091448 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:48.091729 (+ 281us) client_negotiation.cc:769] Sending connection context
0810 00:51:48.091971 (+ 242us) client_negotiation.cc:240] Negotiation successful
0810 00:51:48.092386 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":243,"threads_started":1}
I20260810 00:51:48.094900 15782 heartbeater.cc:347] Connected to a master server at 127.10.97.126:35885
I20260810 00:51:48.095188 15782 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:48.096046 15782 heartbeater.cc:511] Master 127.10.97.126:35885 requested a full tablet report, sending...
I20260810 00:51:48.098308 15458 ts_manager.cc:194] Registered new tserver with Master: 2e2adacd8c40462f87fa32ddddb07942 (127.10.97.65:41495)
I20260810 00:51:48.100332 15458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:57441
WARNING: no policy specified for kudu/127.10.97.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
I20260810 00:51:48.128122 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:35885
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:48.434002 15789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:48.434281 15789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:48.434420 15789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:48.441238 15789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:48.441344 15789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:48.441497 15789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:51:48.450484 15789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:35885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:48.451630 15789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:48.452848 15789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:48.465057 15796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:48.465016 15797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:48.465092 15802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:48.465811 15789 server_base.cc:1034] running on GCE node
I20260810 00:51:48.466354 15789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:48.467161 15789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:48.468348 15789 hybrid_clock.cc:648] HybridClock initialized: now 1786323108468292 us; error 47 us; skew 500 ppm
I20260810 00:51:48.468619 15789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:48 dist-test-slave-jlzn krb5kdc[15393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323108, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:48.471488 15789 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:51:48.472831 15789 webserver.cc:460] Webserver started at http://127.10.97.66:34235/ using document root <none> and password file <none>
I20260810 00:51:48.473398 15789 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:48.473496 15789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:48.473728 15789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:48.476342 15789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "2c702b8c5e284fae8d25a07b9f617f0c"
format_stamp: "Formatted at 2026-08-10 00:51:48 on dist-test-slave-jlzn"
server_key: "2acd8501b8fe8b96eb32486cc9aaafc9"
server_key_iv: "e491787030c4cbfd19fb7b0536c2836e"
server_key_version: "encryptionkey@0"
I20260810 00:51:48.477042 15789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "2c702b8c5e284fae8d25a07b9f617f0c"
format_stamp: "Formatted at 2026-08-10 00:51:48 on dist-test-slave-jlzn"
server_key: "2acd8501b8fe8b96eb32486cc9aaafc9"
server_key_iv: "e491787030c4cbfd19fb7b0536c2836e"
server_key_version: "encryptionkey@0"
I20260810 00:51:48.481535 15789 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:51:48.484572 15813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:48.485821 15789 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:51:48.485963 15789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "2c702b8c5e284fae8d25a07b9f617f0c"
format_stamp: "Formatted at 2026-08-10 00:51:48 on dist-test-slave-jlzn"
server_key: "2acd8501b8fe8b96eb32486cc9aaafc9"
server_key_iv: "e491787030c4cbfd19fb7b0536c2836e"
server_key_version: "encryptionkey@0"
I20260810 00:51:48.486132 15789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:48.501842 15789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:48.505764 15789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:48.506016 15789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:48.506836 15789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:48.508061 15789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:48.508136 15789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:48.508257 15789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:48.508316 15789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:48.538844 15789 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:42625
I20260810 00:51:48.538897 15994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:42625 every 8 connection(s)
I20260810 00:51:48.540030 15789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:51:48.542814 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 15789
I20260810 00:51:48.543030 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:51:48.543584 10629 external_mini_cluster.cc:1428] Setting key 00e7af2b92d4a1bcc1186246e38085e3
Aug 10 00:51:48 dist-test-slave-jlzn krb5kdc[15393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323108, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:48.558701 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:48.542222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:38299 (local address 127.10.97.126:35885)
0810 00:51:48.542394 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:48.542399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:48.543593 (+ 1194us) server_negotiation.cc:407] Connection header received
0810 00:51:48.545814 (+ 2221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:48.545819 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:48.545897 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:48.545980 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:48.547393 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:48.548140 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:48.549621 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:48.550029 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:48.550145 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:48.553454 (+ 3309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:48.553459 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:48.553462 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:48.553493 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:48.555996 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:48.556622 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:48.556628 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:48.556631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:48.556756 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:48.557148 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:48.557152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:48.557154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:48.557399 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:48.557523 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:48.558128 (+ 605us) server_negotiation.cc:299] Negotiation successful
0810 00:51:48.558356 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 00:51:48.559528 15997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:48.542684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:35885 (local address 127.10.97.66:38299)
0810 00:51:48.543438 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:48.543482 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:48.545529 (+ 2047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:48.546233 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:48.546241 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:48.546708 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:48.547236 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:48.547253 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:48.548287 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:48.548293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:48.549484 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:48.549496 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:48.550247 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:48.550254 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:48.550469 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:48.551062 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:48.551090 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:48.553267 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:48.556118 (+ 2851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:48.556126 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:48.556140 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:48.556513 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:48.556857 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:48.556862 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:48.556866 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:48.557036 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:48.557519 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:48.557526 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:48.557815 (+ 289us) client_negotiation.cc:769] Sending connection context
0810 00:51:48.558036 (+ 221us) client_negotiation.cc:240] Negotiation successful
0810 00:51:48.558514 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":329,"threads_started":1}
I20260810 00:51:48.560789 15995 heartbeater.cc:347] Connected to a master server at 127.10.97.126:35885
I20260810 00:51:48.561048 15995 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:48.561720 15995 heartbeater.cc:511] Master 127.10.97.126:35885 requested a full tablet report, sending...
I20260810 00:51:48.563019 15458 ts_manager.cc:194] Registered new tserver with Master: 2c702b8c5e284fae8d25a07b9f617f0c (127.10.97.66:42625)
WARNING: no policy specified for kudu/127.10.97.67@KRBTEST.COM; defaulting to no policy
I20260810 00:51:48.563885 15458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:38299
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
I20260810 00:51:48.586434 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:35885
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:48.894572 16000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:48.894847 16000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:48.894979 16000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:48.901876 16000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:48.901983 16000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:48.902127 16000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:51:48.911201 16000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:35885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:48.912348 16000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:48.913552 16000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:48.925452 16011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:48.925524 16007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:48.925496 16008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:48.926431 16000 server_base.cc:1034] running on GCE node
I20260810 00:51:48.926966 16000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:48.927779 16000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:48.928993 16000 hybrid_clock.cc:648] HybridClock initialized: now 1786323108928946 us; error 41 us; skew 500 ppm
I20260810 00:51:48.929265 16000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:48 dist-test-slave-jlzn krb5kdc[15393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323108, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:48.932204 16000 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:51:48.933439 16000 webserver.cc:460] Webserver started at http://127.10.97.67:41955/ using document root <none> and password file <none>
I20260810 00:51:48.934038 16000 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:48.934154 16000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:48.934427 16000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:48.937155 16000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "c13bd57a5f2d479d873e79dc16243a45"
format_stamp: "Formatted at 2026-08-10 00:51:48 on dist-test-slave-jlzn"
server_key: "045c8be1488c5123d4b8fd94a9cc49b3"
server_key_iv: "7393cae7983a45e468996c20d6efebbf"
server_key_version: "encryptionkey@0"
I20260810 00:51:48.937885 16000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "c13bd57a5f2d479d873e79dc16243a45"
format_stamp: "Formatted at 2026-08-10 00:51:48 on dist-test-slave-jlzn"
server_key: "045c8be1488c5123d4b8fd94a9cc49b3"
server_key_iv: "7393cae7983a45e468996c20d6efebbf"
server_key_version: "encryptionkey@0"
I20260810 00:51:48.942461 16000 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 00:51:48.945577 16021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:48.946741 16000 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260810 00:51:48.946905 16000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "c13bd57a5f2d479d873e79dc16243a45"
format_stamp: "Formatted at 2026-08-10 00:51:48 on dist-test-slave-jlzn"
server_key: "045c8be1488c5123d4b8fd94a9cc49b3"
server_key_iv: "7393cae7983a45e468996c20d6efebbf"
server_key_version: "encryptionkey@0"
I20260810 00:51:48.947127 16000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:48.980142 16000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:48.984205 16000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:48.984467 16000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:48.985318 16000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:48.986696 16000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:48.986768 16000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:48.986861 16000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:48.986904 16000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:49.021317 16000 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:38331
I20260810 00:51:49.021334 16198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:38331 every 8 connection(s)
I20260810 00:51:49.022526 16000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:51:49.031697 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 16000
I20260810 00:51:49.031908 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:51:49.032480 10629 external_mini_cluster.cc:1428] Setting key 2e76a1cb62a67b09fe92d7be83e66399
Aug 10 00:51:49 dist-test-slave-jlzn krb5kdc[15393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323108, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:49.040213 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:49.024861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:50365 (local address 127.10.97.126:35885)
0810 00:51:49.025027 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:49.025032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:49.026125 (+ 1093us) server_negotiation.cc:407] Connection header received
0810 00:51:49.027822 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:49.027829 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:49.027908 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:49.027993 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:49.029324 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:49.030042 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:49.031563 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:49.032016 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:49.032107 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:49.035421 (+ 3314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:49.035426 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:49.035429 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:49.035461 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:49.037735 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:49.038332 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:49.038335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:49.038338 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:49.038453 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:49.038792 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:49.038795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:49.038797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:49.039051 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:49.039151 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:49.039772 (+ 621us) server_negotiation.cc:299] Negotiation successful
0810 00:51:49.039972 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 00:51:49.041078 16203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:49.025330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:35885 (local address 127.10.97.67:50365)
0810 00:51:49.025978 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:49.026015 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:49.027572 (+ 1557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:49.028187 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:49.028194 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:49.028655 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:49.029196 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:49.029214 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:49.030178 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:49.030182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:49.031432 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:49.031446 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:49.032256 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:49.032263 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:49.032552 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:49.033203 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:49.033230 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:49.035262 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:49.037847 (+ 2585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:49.037853 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:49.037863 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:49.038244 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:49.038532 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:49.038535 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:49.038537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:49.038716 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:49.039141 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:49.039146 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:49.039439 (+ 293us) client_negotiation.cc:769] Sending connection context
0810 00:51:49.039704 (+ 265us) client_negotiation.cc:240] Negotiation successful
0810 00:51:49.040082 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":257,"threads_started":1}
I20260810 00:51:49.042243 16199 heartbeater.cc:347] Connected to a master server at 127.10.97.126:35885
I20260810 00:51:49.042493 16199 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:49.043154 16199 heartbeater.cc:511] Master 127.10.97.126:35885 requested a full tablet report, sending...
I20260810 00:51:49.044461 15458 ts_manager.cc:194] Registered new tserver with Master: c13bd57a5f2d479d873e79dc16243a45 (127.10.97.67:38331)
I20260810 00:51:49.045233 15458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:50365
I20260810 00:51:49.047240 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 00:51:49 dist-test-slave-jlzn krb5kdc[15393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323109, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 00:51:49 dist-test-slave-jlzn krb5kdc[15393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323109, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:49.097199 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:49.083913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43838 (local address 127.10.97.126:35885)
0810 00:51:49.084090 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:49.084096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:49.084545 (+ 449us) server_negotiation.cc:407] Connection header received
0810 00:51:49.084735 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:49.084740 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:49.084823 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:49.084909 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:49.086367 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:49.087096 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:49.088361 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:49.088796 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:49.088913 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:49.092581 (+ 3668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:49.092586 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:49.092589 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:49.092623 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:49.094790 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:49.095377 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:49.095381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:49.095383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:49.095507 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:49.095896 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:49.095901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:49.095904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:49.096211 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:49.096423 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:49.096734 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 00:51:49.096937 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260810 00:51:49.097738 15458 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43838
I20260810 00:51:49.098892 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 15573
I20260810 00:51:49.103699 15782 heartbeater.cc:503] Master 127.10.97.126:35885 was elected leader, sending a full tablet report...
Aug 10 00:51:49 dist-test-slave-jlzn krb5kdc[15393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323109, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:51:49.113267 16222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:49.099961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46370 (local address 127.10.97.65:41495)
0810 00:51:49.100502 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:49.100508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:49.100533 (+ 25us) server_negotiation.cc:407] Connection header received
0810 00:51:49.100587 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:49.100593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:49.100779 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:49.100902 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:49.101871 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:49.102824 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:49.103959 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:49.104656 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:49.104750 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:49.108215 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:49.108226 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:49.108238 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:49.108267 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:49.111023 (+ 2756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:49.111594 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:49.111600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:49.111602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:49.111715 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:49.112027 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:49.112030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:49.112032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:49.112348 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:49.112564 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:49.112730 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 00:51:49.112908 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":272,"threads_started":1}
W20260810 00:51:49.114373 15770 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46370
W20260810 00:51:49.115348 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 15573: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:51:49.115502 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 15573
I20260810 00:51:49.137804 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 15789
Aug 10 00:51:49 dist-test-slave-jlzn krb5kdc[15393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323109, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.97.66@KRBTEST.COM
I20260810 00:51:49.152845 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:49.138609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34390 (local address 127.10.97.66:42625)
0810 00:51:49.139059 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:49.139066 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:49.139090 (+ 24us) server_negotiation.cc:407] Connection header received
0810 00:51:49.139158 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:49.139164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:49.139360 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:49.139496 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:49.140778 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:49.141745 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:49.142956 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:49.143494 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:49.143603 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:49.147436 (+ 3833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:49.147447 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:49.147460 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:49.147492 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:49.150145 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:49.150776 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:49.150783 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:49.150786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:49.150913 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:49.151288 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:49.151293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:49.151297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:49.151641 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:49.151960 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:49.152249 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 00:51:49.152521 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":213,"threads_started":1}
W20260810 00:51:49.154106 15987 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34390
W20260810 00:51:49.155138 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 15789: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:51:49.155296 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 15789
I20260810 00:51:49.177829 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 16000
Aug 10 00:51:49 dist-test-slave-jlzn krb5kdc[15393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323109, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.97.67@KRBTEST.COM
I20260810 00:51:49.192942 16224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:49.178711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48092 (local address 127.10.97.67:38331)
0810 00:51:49.179267 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:49.179274 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:49.179302 (+ 28us) server_negotiation.cc:407] Connection header received
0810 00:51:49.179360 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:49.179365 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:49.179569 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:49.179717 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:49.181061 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:49.182099 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:49.183292 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:49.183915 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:49.184025 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:49.188141 (+ 4116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:49.188152 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:49.188167 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:49.188200 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:49.190693 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:49.191261 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:49.191268 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:49.191270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:49.191384 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:49.191658 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:49.191661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:49.191664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:49.192038 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:49.192277 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:49.192468 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 00:51:49.192656 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":242,"threads_started":1}
W20260810 00:51:49.194152 16193 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48092
W20260810 00:51:49.195191 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 16000: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:51:49.195317 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 16000
I20260810 00:51:49.217659 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 15413
I20260810 00:51:49.218705 15512 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:49.315591 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 15413
2026-08-10T00:51:49Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3545 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:51:49 dist-test-slave-jlzn krb5kdc[16229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:51:49 dist-test-slave-jlzn krb5kdc[16229](info): set up 2 sockets
Aug 10 00:51:49 dist-test-slave-jlzn krb5kdc[16229](info): commencing operation
krb5kdc: starting...
W20260810 00:51:50.478708 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:51:50 dist-test-slave-jlzn krb5kdc[16229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:51:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:51:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:51:50.658993 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:40465
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:33479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:40465
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:50.968456 16250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:50.968734 16250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:50.968828 16250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:50.975587 16250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:51:50.975678 16250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:50.975730 16250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:50.975788 16250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:51:50.975849 16250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:51:50.984933 16250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:40465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:40465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:50.985975 16250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:50.987138 16250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:50.997809 16258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:50.998260 16261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:50.998329 16256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:50.998517 16250 server_base.cc:1034] running on GCE node
I20260810 00:51:50.999228 16250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:51.000639 16250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:51.001855 16250 hybrid_clock.cc:648] HybridClock initialized: now 1786323111001803 us; error 38 us; skew 500 ppm
I20260810 00:51:51.002177 16250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:51 dist-test-slave-jlzn krb5kdc[16229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323111, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:51.005123 16250 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:51:51.006417 16250 webserver.cc:460] Webserver started at http://127.10.97.126:36493/ using document root <none> and password file <none>
I20260810 00:51:51.007001 16250 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:51.007126 16250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:51.007411 16250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:51.010165 16250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "4a0628fb238d48978c9c8d6c380be2be"
format_stamp: "Formatted at 2026-08-10 00:51:51 on dist-test-slave-jlzn"
server_key: "28d97547685ea91b794cd34845e75a05"
server_key_iv: "0b90a27ca2c118642f5732869f121815"
server_key_version: "encryptionkey@0"
I20260810 00:51:51.010878 16250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "4a0628fb238d48978c9c8d6c380be2be"
format_stamp: "Formatted at 2026-08-10 00:51:51 on dist-test-slave-jlzn"
server_key: "28d97547685ea91b794cd34845e75a05"
server_key_iv: "0b90a27ca2c118642f5732869f121815"
server_key_version: "encryptionkey@0"
I20260810 00:51:51.015522 16250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260810 00:51:51.018648 16276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:51.019773 16250 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260810 00:51:51.019951 16250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "4a0628fb238d48978c9c8d6c380be2be"
format_stamp: "Formatted at 2026-08-10 00:51:51 on dist-test-slave-jlzn"
server_key: "28d97547685ea91b794cd34845e75a05"
server_key_iv: "0b90a27ca2c118642f5732869f121815"
server_key_version: "encryptionkey@0"
I20260810 00:51:51.020125 16250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:51.035211 16250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:51.038965 16250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:51.039208 16250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:51.054960 16250 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:40465
I20260810 00:51:51.054979 16350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:40465 every 8 connection(s)
I20260810 00:51:51.056205 16250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:51:51.059561 16352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:51.063184 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 16250
I20260810 00:51:51.063366 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:51:51.064055 10629 external_mini_cluster.cc:1428] Setting key 02f35f6d427483315366f9626fcd702f
I20260810 00:51:51.070797 16352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be: Bootstrap starting.
I20260810 00:51:51.074221 16352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:51.075376 16352 log.cc:824] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:51.077942 16352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be: No bootstrap required, opened a new log
Aug 10 00:51:51 dist-test-slave-jlzn krb5kdc[16229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:51.086263 16352 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0628fb238d48978c9c8d6c380be2be" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 40465 } }
I20260810 00:51:51.086508 16358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:51.065772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51700 (local address 127.10.97.126:40465)
0810 00:51:51.066543 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:51.066558 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:51.066610 (+ 52us) server_negotiation.cc:407] Connection header received
0810 00:51:51.069207 (+ 2597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:51.069234 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:51.069676 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:51.070061 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:51.071343 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:51.073337 (+ 1994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:51.074580 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:51.075577 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:51.075756 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:51.079865 (+ 4109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:51.079874 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:51.079891 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:51.079927 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:51.082833 (+ 2906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:51.083551 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:51.083557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:51.083561 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:51.083738 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:51.084134 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:51.084138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:51.084140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:51.084644 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:51.084856 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:51.085295 (+ 439us) server_negotiation.cc:299] Negotiation successful
0810 00:51:51.085597 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":322,"threads_started":1}
I20260810 00:51:51.086594 16352 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:51.086683 16352 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0628fb238d48978c9c8d6c380be2be, State: Initialized, Role: FOLLOWER
I20260810 00:51:51.087271 16352 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0628fb238d48978c9c8d6c380be2be" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 40465 } }
I20260810 00:51:51.087494 16352 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:51:51.087628 16352 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:51:51.087795 16352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:51.090313 16352 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0628fb238d48978c9c8d6c380be2be" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 40465 } }
I20260810 00:51:51.090778 16352 leader_election.cc:302] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a0628fb238d48978c9c8d6c380be2be; no voters:
I20260810 00:51:51.091383 16352 leader_election.cc:288] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:51:51.091485 16362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:51.092139 16362 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be [term 1 LEADER]: Becoming Leader. State: Replica: 4a0628fb238d48978c9c8d6c380be2be, State: Running, Role: LEADER
I20260810 00:51:51.092628 16362 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0628fb238d48978c9c8d6c380be2be" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 40465 } }
I20260810 00:51:51.093185 16352 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:51:51.095192 16365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a0628fb238d48978c9c8d6c380be2be. Latest consensus state: current_term: 1 leader_uuid: "4a0628fb238d48978c9c8d6c380be2be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0628fb238d48978c9c8d6c380be2be" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 40465 } } }
I20260810 00:51:51.095165 16364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a0628fb238d48978c9c8d6c380be2be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0628fb238d48978c9c8d6c380be2be" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 40465 } } }
I20260810 00:51:51.095589 16365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be [sys.catalog]: This master's current role is: LEADER
I20260810 00:51:51.095589 16364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be [sys.catalog]: This master's current role is: LEADER
I20260810 00:51:51.096447 16374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:51:51.101298 16374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:51:51.109295 16374 catalog_manager.cc:1347] Generated new cluster ID: 3618ed7a80d34cbdad46d87c90f3638b
I20260810 00:51:51.109390 16374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:51:51.145704 16374 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:51:51.147473 16374 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:51:51.166102 16374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a0628fb238d48978c9c8d6c380be2be: Generated new TSK 0
I20260810 00:51:51.167030 16374 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
I20260810 00:51:51.220655 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:40465
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:33479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:51.530440 16391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:51.530697 16391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:51.530794 16391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:51.537724 16391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:51.537822 16391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:51.537933 16391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:51:51.547170 16391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:40465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:51.548266 16391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:51.549417 16391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:51.561352 16399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:51.561431 16400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:51.561396 16404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:51.562188 16391 server_base.cc:1034] running on GCE node
I20260810 00:51:51.562714 16391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:51.563534 16391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:51.564720 16391 hybrid_clock.cc:648] HybridClock initialized: now 1786323111564673 us; error 41 us; skew 500 ppm
I20260810 00:51:51.564997 16391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:51 dist-test-slave-jlzn krb5kdc[16229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323111, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:51.567875 16391 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:51:51.569175 16391 webserver.cc:460] Webserver started at http://127.10.97.65:44801/ using document root <none> and password file <none>
I20260810 00:51:51.569736 16391 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:51.569836 16391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:51.570075 16391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:51.572768 16391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "d906b720891f4c189d085318a9e6db1d"
format_stamp: "Formatted at 2026-08-10 00:51:51 on dist-test-slave-jlzn"
server_key: "a842ed3cdb4450eb205da9610fd1ce12"
server_key_iv: "62c6ca3a684fe76cacd2b8eaa8a572a2"
server_key_version: "encryptionkey@0"
I20260810 00:51:51.573472 16391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "d906b720891f4c189d085318a9e6db1d"
format_stamp: "Formatted at 2026-08-10 00:51:51 on dist-test-slave-jlzn"
server_key: "a842ed3cdb4450eb205da9610fd1ce12"
server_key_iv: "62c6ca3a684fe76cacd2b8eaa8a572a2"
server_key_version: "encryptionkey@0"
I20260810 00:51:51.578125 16391 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:51:51.581234 16419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:51.582383 16391 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:51:51.582511 16391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "d906b720891f4c189d085318a9e6db1d"
format_stamp: "Formatted at 2026-08-10 00:51:51 on dist-test-slave-jlzn"
server_key: "a842ed3cdb4450eb205da9610fd1ce12"
server_key_iv: "62c6ca3a684fe76cacd2b8eaa8a572a2"
server_key_version: "encryptionkey@0"
I20260810 00:51:51.582660 16391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:51.602077 16391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:51.605562 16391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:51.605793 16391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:51.606595 16391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:51.607777 16391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:51.607882 16391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:51.607978 16391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:51.608068 16391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:51.636693 16391 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:33033
I20260810 00:51:51.636777 16603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:33033 every 8 connection(s)
I20260810 00:51:51.637802 16391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:51:51.647159 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 16391
I20260810 00:51:51.647344 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:51:51.648017 10629 external_mini_cluster.cc:1428] Setting key 8268c716f16e7ac10a77834b25fbe438
Aug 10 00:51:51 dist-test-slave-jlzn krb5kdc[16229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323111, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:51.656322 16608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:51.639988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:44761 (local address 127.10.97.126:40465)
0810 00:51:51.640364 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:51.640369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:51.641214 (+ 845us) server_negotiation.cc:407] Connection header received
0810 00:51:51.642937 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:51.642943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:51.643020 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:51.643116 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:51.644558 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:51.645283 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:51.646756 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:51.647175 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:51.647284 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:51.650586 (+ 3302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:51.650592 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:51.650596 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:51.650632 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:51.653478 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:51.654124 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:51.654129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:51.654140 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:51.654282 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:51.654634 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:51.654639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:51.654642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:51.654963 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:51.655105 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:51.655760 (+ 655us) server_negotiation.cc:299] Negotiation successful
0810 00:51:51.655991 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":190,"threads_started":1}
I20260810 00:51:51.657065 16610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:51.640401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:40465 (local address 127.10.97.65:44761)
0810 00:51:51.641059 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:51.641093 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:51.642684 (+ 1591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:51.643363 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:51.643371 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:51.643866 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:51.644391 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:51.644408 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:51.645429 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:51.645432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:51.646613 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:51.646626 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:51.647389 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:51.647396 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:51.647597 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:51.648192 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:51.648220 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:51.650376 (+ 2156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:51.653598 (+ 3222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:51.653604 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:51.653614 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:51.654002 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:51.654366 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:51.654369 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:51.654371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:51.654531 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:51.655068 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:51.655075 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:51.655397 (+ 322us) client_negotiation.cc:769] Sending connection context
0810 00:51:51.655635 (+ 238us) client_negotiation.cc:240] Negotiation successful
0810 00:51:51.656058 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":261,"threads_started":1}
I20260810 00:51:51.658412 16604 heartbeater.cc:347] Connected to a master server at 127.10.97.126:40465
I20260810 00:51:51.658702 16604 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:51.659492 16604 heartbeater.cc:511] Master 127.10.97.126:40465 requested a full tablet report, sending...
I20260810 00:51:51.661659 16296 ts_manager.cc:194] Registered new tserver with Master: d906b720891f4c189d085318a9e6db1d (127.10.97.65:33033)
I20260810 00:51:51.663552 16296 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:44761
WARNING: no policy specified for kudu/127.10.97.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
I20260810 00:51:51.694415 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:40465
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:33479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:52.002143 16614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:52.002394 16614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:52.002523 16614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:52.009387 16614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:52.009495 16614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:52.009645 16614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:51:52.018699 16614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:40465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:52.019769 16614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:52.021013 16614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:52.033233 16622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:52.033267 16624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:52.033377 16626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:52.034162 16614 server_base.cc:1034] running on GCE node
I20260810 00:51:52.034735 16614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:52.035521 16614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:52.036703 16614 hybrid_clock.cc:648] HybridClock initialized: now 1786323112036653 us; error 41 us; skew 500 ppm
I20260810 00:51:52.036972 16614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:52 dist-test-slave-jlzn krb5kdc[16229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323112, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:52.039815 16614 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:51:52.041064 16614 webserver.cc:460] Webserver started at http://127.10.97.66:43685/ using document root <none> and password file <none>
I20260810 00:51:52.041659 16614 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:52.041775 16614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:52.042047 16614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:52.044780 16614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "b7b47358b4a94e689e7be99557f15920"
format_stamp: "Formatted at 2026-08-10 00:51:52 on dist-test-slave-jlzn"
server_key: "7d793e2131534f1cb8ac476931eee525"
server_key_iv: "79ea5c9f5b33807d715177d33aca0291"
server_key_version: "encryptionkey@0"
I20260810 00:51:52.045498 16614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "b7b47358b4a94e689e7be99557f15920"
format_stamp: "Formatted at 2026-08-10 00:51:52 on dist-test-slave-jlzn"
server_key: "7d793e2131534f1cb8ac476931eee525"
server_key_iv: "79ea5c9f5b33807d715177d33aca0291"
server_key_version: "encryptionkey@0"
I20260810 00:51:52.050076 16614 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:51:52.053241 16636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:52.054442 16614 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260810 00:51:52.054595 16614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "b7b47358b4a94e689e7be99557f15920"
format_stamp: "Formatted at 2026-08-10 00:51:52 on dist-test-slave-jlzn"
server_key: "7d793e2131534f1cb8ac476931eee525"
server_key_iv: "79ea5c9f5b33807d715177d33aca0291"
server_key_version: "encryptionkey@0"
I20260810 00:51:52.054816 16614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:52.068528 16614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:52.072999 16614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:52.073262 16614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:52.074231 16614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:52.075608 16614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:52.075688 16614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:52.075774 16614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:52.075847 16614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260810 00:51:52.104803 16614 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:37133
I20260810 00:51:52.104861 16803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:37133 every 8 connection(s)
I20260810 00:51:52.105908 16614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:51:52.109836 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 16614
I20260810 00:51:52.110001 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:51:52.110534 10629 external_mini_cluster.cc:1428] Setting key 5753140b1b79653692866d431bc4cf0f
Aug 10 00:51:52 dist-test-slave-jlzn krb5kdc[16229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323112, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:52.125679 16608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:52.108170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:43597 (local address 127.10.97.126:40465)
0810 00:51:52.108338 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:52.108342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:52.109403 (+ 1061us) server_negotiation.cc:407] Connection header received
0810 00:51:52.111685 (+ 2282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:52.111690 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:52.111774 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:52.111890 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:52.113687 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:52.114391 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:52.116407 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:52.116801 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:52.116888 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:52.120407 (+ 3519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:52.120412 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:52.120415 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:52.120445 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:52.122860 (+ 2415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:52.123538 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:52.123543 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:52.123546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:52.123683 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:52.124109 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:52.124114 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:52.124117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:52.124370 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:52.124508 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:52.125147 (+ 639us) server_negotiation.cc:299] Negotiation successful
0810 00:51:52.125361 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 00:51:52.126488 16811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:52.108566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:40465 (local address 127.10.97.66:43597)
0810 00:51:52.109257 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:52.109292 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:52.111404 (+ 2112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:52.112104 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:52.112112 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:52.112773 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:52.113519 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:52.113543 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:52.114542 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:52.114547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:52.116284 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:52.116304 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:52.117000 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:52.117006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:52.117267 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:52.117903 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:52.117929 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:52.120211 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:52.122989 (+ 2778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:52.122998 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:52.123012 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:52.123406 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:52.123788 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:52.123793 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:52.123797 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:52.124000 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:52.124491 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:52.124499 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:52.124789 (+ 290us) client_negotiation.cc:769] Sending connection context
0810 00:51:52.125003 (+ 214us) client_negotiation.cc:240] Negotiation successful
0810 00:51:52.125455 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":281,"threads_started":1}
I20260810 00:51:52.127769 16804 heartbeater.cc:347] Connected to a master server at 127.10.97.126:40465
I20260810 00:51:52.128065 16804 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:52.128698 16804 heartbeater.cc:511] Master 127.10.97.126:40465 requested a full tablet report, sending...
I20260810 00:51:52.130054 16296 ts_manager.cc:194] Registered new tserver with Master: b7b47358b4a94e689e7be99557f15920 (127.10.97.66:37133)
I20260810 00:51:52.131138 16296 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:43597
WARNING: no policy specified for kudu/127.10.97.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
I20260810 00:51:52.155032 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:40465
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:33479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:52.463647 16815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:52.463909 16815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:52.464004 16815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:52.470854 16815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:52.470950 16815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:52.471094 16815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:51:52.480381 16815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:40465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:52.481467 16815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:52.482685 16815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:52.494805 16823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:52.494849 16826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:52.494966 16822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:52.495757 16815 server_base.cc:1034] running on GCE node
I20260810 00:51:52.496340 16815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:52.497144 16815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:52.498342 16815 hybrid_clock.cc:648] HybridClock initialized: now 1786323112498290 us; error 42 us; skew 500 ppm
I20260810 00:51:52.498625 16815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:52 dist-test-slave-jlzn krb5kdc[16229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323112, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:52.504464 16815 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:51:52.505749 16815 webserver.cc:460] Webserver started at http://127.10.97.67:37099/ using document root <none> and password file <none>
I20260810 00:51:52.506371 16815 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:52.506486 16815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:52.506772 16815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:52.509629 16815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "d65236248db8489688651f7ad907d8c9"
format_stamp: "Formatted at 2026-08-10 00:51:52 on dist-test-slave-jlzn"
server_key: "d009a784795b0df0bff84e89345c153b"
server_key_iv: "f30e876ebc4c22e74e660261b6f833e6"
server_key_version: "encryptionkey@0"
I20260810 00:51:52.510377 16815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "d65236248db8489688651f7ad907d8c9"
format_stamp: "Formatted at 2026-08-10 00:51:52 on dist-test-slave-jlzn"
server_key: "d009a784795b0df0bff84e89345c153b"
server_key_iv: "f30e876ebc4c22e74e660261b6f833e6"
server_key_version: "encryptionkey@0"
I20260810 00:51:52.515168 16815 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 00:51:52.518420 16840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:52.519637 16815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 00:51:52.519783 16815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "d65236248db8489688651f7ad907d8c9"
format_stamp: "Formatted at 2026-08-10 00:51:52 on dist-test-slave-jlzn"
server_key: "d009a784795b0df0bff84e89345c153b"
server_key_iv: "f30e876ebc4c22e74e660261b6f833e6"
server_key_version: "encryptionkey@0"
I20260810 00:51:52.520015 16815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:52.533437 16815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:52.537375 16815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:52.537647 16815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:52.538534 16815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:52.540479 16815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:52.540563 16815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:52.540683 16815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:52.540740 16815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:52.570293 16815 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:39787
I20260810 00:51:52.570370 17038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:39787 every 8 connection(s)
I20260810 00:51:52.571486 16815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:51:52.580773 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 16815
I20260810 00:51:52.580920 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:51:52.581476 10629 external_mini_cluster.cc:1428] Setting key fa238dae537127da95d264a31e763f11
Aug 10 00:51:52 dist-test-slave-jlzn krb5kdc[16229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323112, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:52.589583 16608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:52.573735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:46601 (local address 127.10.97.126:40465)
0810 00:51:52.573892 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:52.573896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:52.575006 (+ 1110us) server_negotiation.cc:407] Connection header received
0810 00:51:52.576721 (+ 1715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:52.576726 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:52.576798 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:52.576883 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:52.578214 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:52.578966 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:52.580481 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:52.580938 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:52.581055 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:52.584812 (+ 3757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:52.584817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:52.584820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:52.584851 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:52.587251 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:52.587855 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:52.587859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:52.587861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:52.587974 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:52.588313 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:52.588315 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:52.588317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:52.588566 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:52.588672 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:52.589200 (+ 528us) server_negotiation.cc:299] Negotiation successful
0810 00:51:52.589358 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 00:51:52.590379 17042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:52.574164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:40465 (local address 127.10.97.67:46601)
0810 00:51:52.574844 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:52.574878 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:52.576485 (+ 1607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:52.577057 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:52.577063 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:52.577527 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:52.578060 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:52.578077 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:52.579080 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:52.579083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:52.580334 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:52.580347 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:52.581160 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:52.581167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:52.581442 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:52.582077 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:52.582098 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:52.584601 (+ 2503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:52.587364 (+ 2763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:52.587370 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:52.587379 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:52.587747 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:52.588050 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:52.588053 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:52.588055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:52.588234 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:52.588654 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:52.588659 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:52.588920 (+ 261us) client_negotiation.cc:769] Sending connection context
0810 00:51:52.589114 (+ 194us) client_negotiation.cc:240] Negotiation successful
0810 00:51:52.589444 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":263,"threads_started":1}
I20260810 00:51:52.591486 17039 heartbeater.cc:347] Connected to a master server at 127.10.97.126:40465
I20260810 00:51:52.591733 17039 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:52.592374 17039 heartbeater.cc:511] Master 127.10.97.126:40465 requested a full tablet report, sending...
I20260810 00:51:52.593617 16296 ts_manager.cc:194] Registered new tserver with Master: d65236248db8489688651f7ad907d8c9 (127.10.97.67:39787)
I20260810 00:51:52.594411 16296 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:46601
I20260810 00:51:52.596613 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 00:51:52 dist-test-slave-jlzn krb5kdc[16229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 00:51:52 dist-test-slave-jlzn krb5kdc[16229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:51:52.647625 17051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:52.632601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57872 (local address 127.10.97.65:33033)
0810 00:51:52.633171 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:52.633177 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:52.633204 (+ 27us) server_negotiation.cc:407] Connection header received
0810 00:51:52.633269 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:52.633276 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:52.633474 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:52.633595 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:52.634864 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:52.635861 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:52.637569 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:52.638121 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:52.638217 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:52.642731 (+ 4514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:52.642742 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:52.642753 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:52.642787 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:52.645435 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:52.645971 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:52.645976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:52.645979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:52.646098 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:52.646388 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:52.646391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:52.646393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:52.646709 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:52.646913 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:52.647096 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 00:51:52.647279 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":302,"threads_started":1}
Aug 10 00:51:52 dist-test-slave-jlzn krb5kdc[16229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:52.663169 16608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:52.651435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38142 (local address 127.10.97.126:40465)
0810 00:51:52.651656 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:52.651662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:52.651785 (+ 123us) server_negotiation.cc:407] Connection header received
0810 00:51:52.651955 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:52.651959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:52.652023 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:52.652091 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:52.653165 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:52.653877 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:52.655010 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:52.655414 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:52.655500 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:52.658822 (+ 3322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:52.658827 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:52.658830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:52.658859 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:52.661201 (+ 2342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:52.661716 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:52.661719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:52.661721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:52.661833 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:52.662079 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:52.662081 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:52.662083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:52.662354 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:52.662525 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:52.662734 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 00:51:52.662881 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260810 00:51:52.663738 16296 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38142
I20260810 00:51:52.664701 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 16391
I20260810 00:51:52.666323 16604 heartbeater.cc:503] Master 127.10.97.126:40465 was elected leader, sending a full tablet report...
I20260810 00:51:52.675426 17051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:52.665414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57878 (local address 127.10.97.65:33033)
0810 00:51:52.665538 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:52.665542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:52.665840 (+ 298us) server_negotiation.cc:407] Connection header received
0810 00:51:52.665947 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:52.665950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:52.666029 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:52.666101 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:52.667370 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:52.668056 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:52.669159 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:52.669529 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:52.669612 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:52.671408 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:52.671413 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:52.671416 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:52.671442 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:52.673629 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:52.674147 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:52.674150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:52.674152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:52.674250 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:52.674497 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:52.674500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:52.674502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:52.674707 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:52.674860 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:52.675063 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 00:51:52.675217 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 00:51:52.675951 16599 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:52.756146 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 16391
I20260810 00:51:52.779981 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 16614
Aug 10 00:51:52 dist-test-slave-jlzn krb5kdc[16229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.66@KRBTEST.COM
I20260810 00:51:52.795162 17060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:52.780819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34958 (local address 127.10.97.66:37133)
0810 00:51:52.781395 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:52.781403 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:52.781430 (+ 27us) server_negotiation.cc:407] Connection header received
0810 00:51:52.781493 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:52.781499 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:52.781717 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:52.781846 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:52.782945 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:52.783990 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:52.785245 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:52.785781 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:52.785881 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:52.789690 (+ 3809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:52.789702 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:52.789716 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:52.789748 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:52.792706 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:52.793352 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:52.793360 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:52.793362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:52.793480 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:52.793819 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:52.793822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:52.793824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:52.794197 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:52.794427 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:52.794688 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 00:51:52.794874 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":288,"threads_started":1}
I20260810 00:51:52.796448 16798 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:52.876853 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 16614
I20260810 00:51:52.899597 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 16815
Aug 10 00:51:52 dist-test-slave-jlzn krb5kdc[16229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.67@KRBTEST.COM
I20260810 00:51:52.914618 17063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:52.900607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48498 (local address 127.10.97.67:39787)
0810 00:51:52.901241 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:52.901248 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:52.901276 (+ 28us) server_negotiation.cc:407] Connection header received
0810 00:51:52.901336 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:52.901342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:52.901572 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:52.901699 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:52.902893 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:52.904016 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:52.905214 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:52.905739 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:52.905830 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:52.909727 (+ 3897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:52.909738 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:52.909751 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:52.909782 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:52.912450 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:52.913011 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:52.913016 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:52.913018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:52.913129 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:52.913384 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:52.913387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:52.913389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:52.913689 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:52.913915 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:52.914162 (+ 247us) server_negotiation.cc:299] Negotiation successful
0810 00:51:52.914346 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":324,"threads_started":1}
I20260810 00:51:52.915833 17032 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:52.995558 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 16815
I20260810 00:51:53.015691 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 16250
I20260810 00:51:53.016903 16346 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:53.115892 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 16250
2026-08-10T00:51:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3805 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:51:53 dist-test-slave-jlzn krb5kdc[17076](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:51:53 dist-test-slave-jlzn krb5kdc[17076](info): set up 2 sockets
Aug 10 00:51:53 dist-test-slave-jlzn krb5kdc[17076](info): commencing operation
krb5kdc: starting...
W20260810 00:51:54.304312 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:51:54 dist-test-slave-jlzn krb5kdc[17076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:51:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:51:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:51:54.490378 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:39723
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:35625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:39723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:54.799007 17100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:54.799288 17100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:54.799407 17100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:54.806438 17100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:51:54.806548 17100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:54.806623 17100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:54.806694 17100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:51:54.806756 17100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:51:54.816354 17100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:35625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:39723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:39723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:54.817567 17100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:54.818845 17100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:54.829614 17114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:54.829679 17111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:54.829802 17108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:54.830470 17100 server_base.cc:1034] running on GCE node
I20260810 00:51:54.831197 17100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:54.832477 17100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:54.833711 17100 hybrid_clock.cc:648] HybridClock initialized: now 1786323114833667 us; error 34 us; skew 500 ppm
I20260810 00:51:54.834036 17100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:54 dist-test-slave-jlzn krb5kdc[17076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323114, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:54.836905 17100 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:51:54.837987 17100 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:54.838088 17100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:54.838389 17100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:54.841189 17100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "a169dccfd7d14aa091a1296e6c8cbe1f"
format_stamp: "Formatted at 2026-08-10 00:51:54 on dist-test-slave-jlzn"
server_key: "e10ab5628344b907ae8bf3d713eb83a5"
server_key_iv: "df894b4e84b10786e892956dd4a13c25"
server_key_version: "encryptionkey@0"
I20260810 00:51:54.841941 17100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "a169dccfd7d14aa091a1296e6c8cbe1f"
format_stamp: "Formatted at 2026-08-10 00:51:54 on dist-test-slave-jlzn"
server_key: "e10ab5628344b907ae8bf3d713eb83a5"
server_key_iv: "df894b4e84b10786e892956dd4a13c25"
server_key_version: "encryptionkey@0"
I20260810 00:51:54.846549 17100 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260810 00:51:54.849687 17124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:54.850821 17100 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:51:54.850977 17100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "a169dccfd7d14aa091a1296e6c8cbe1f"
format_stamp: "Formatted at 2026-08-10 00:51:54 on dist-test-slave-jlzn"
server_key: "e10ab5628344b907ae8bf3d713eb83a5"
server_key_iv: "df894b4e84b10786e892956dd4a13c25"
server_key_version: "encryptionkey@0"
I20260810 00:51:54.851166 17100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:54.867167 17100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:54.874089 17100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:54.875847 17100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:54.899497 17100 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:39723
I20260810 00:51:54.899515 17205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:39723 every 8 connection(s)
I20260810 00:51:54.900497 17100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:51:54.904100 17206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:54.907590 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 17100
I20260810 00:51:54.907793 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:51:54.908584 10629 external_mini_cluster.cc:1428] Setting key cb209f48a96e932d84a1d9fd39c1a98f
I20260810 00:51:54.916666 17206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f: Bootstrap starting.
I20260810 00:51:54.920202 17206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:54.921442 17206 log.cc:824] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f: Log is configured to *not* fsync() on all Append() calls
Aug 10 00:51:54 dist-test-slave-jlzn krb5kdc[17076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:54.924193 17206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f: No bootstrap required, opened a new log
I20260810 00:51:54.931365 17210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:54.910471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53810 (local address 127.10.97.126:39723)
0810 00:51:54.911217 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:54.911228 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:54.911266 (+ 38us) server_negotiation.cc:407] Connection header received
0810 00:51:54.912834 (+ 1568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:54.912856 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:54.913291 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:54.913644 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:54.915354 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:54.916636 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:54.918344 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:54.918975 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:54.919098 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:54.923451 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:54.923462 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:54.923478 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:54.923525 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:54.927236 (+ 3711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:54.927883 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:54.927891 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:54.927899 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:54.928115 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:54.928442 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:54.928446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:54.928449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:54.929139 (+ 690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:54.929415 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:54.929693 (+ 278us) server_negotiation.cc:299] Negotiation successful
0810 00:51:54.930093 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":342,"threads_started":1}
I20260810 00:51:54.934178 17206 raft_consensus.cc:348] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a169dccfd7d14aa091a1296e6c8cbe1f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39723 } }
I20260810 00:51:54.934480 17206 raft_consensus.cc:374] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:54.934551 17206 raft_consensus.cc:729] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a169dccfd7d14aa091a1296e6c8cbe1f, State: Initialized, Role: FOLLOWER
I20260810 00:51:54.935015 17206 consensus_queue.cc:260] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a169dccfd7d14aa091a1296e6c8cbe1f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39723 } }
I20260810 00:51:54.935220 17206 raft_consensus.cc:388] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:51:54.935313 17206 raft_consensus.cc:482] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:51:54.935428 17206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:54.937422 17206 raft_consensus.cc:504] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a169dccfd7d14aa091a1296e6c8cbe1f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39723 } }
I20260810 00:51:54.937906 17206 leader_election.cc:302] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a169dccfd7d14aa091a1296e6c8cbe1f; no voters:
I20260810 00:51:54.938570 17206 leader_election.cc:288] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:51:54.938663 17215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:54.939280 17215 raft_consensus.cc:686] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f [term 1 LEADER]: Becoming Leader. State: Replica: a169dccfd7d14aa091a1296e6c8cbe1f, State: Running, Role: LEADER
I20260810 00:51:54.939751 17215 consensus_queue.cc:237] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a169dccfd7d14aa091a1296e6c8cbe1f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39723 } }
I20260810 00:51:54.940471 17206 sys_catalog.cc:564] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:51:54.942451 17220 sys_catalog.cc:455] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a169dccfd7d14aa091a1296e6c8cbe1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a169dccfd7d14aa091a1296e6c8cbe1f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39723 } } }
I20260810 00:51:54.942605 17221 sys_catalog.cc:455] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a169dccfd7d14aa091a1296e6c8cbe1f. Latest consensus state: current_term: 1 leader_uuid: "a169dccfd7d14aa091a1296e6c8cbe1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a169dccfd7d14aa091a1296e6c8cbe1f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39723 } } }
I20260810 00:51:54.942806 17220 sys_catalog.cc:458] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f [sys.catalog]: This master's current role is: LEADER
I20260810 00:51:54.942921 17221 sys_catalog.cc:458] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f [sys.catalog]: This master's current role is: LEADER
I20260810 00:51:54.946439 17237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:51:54.951299 17237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:51:54.959657 17237 catalog_manager.cc:1347] Generated new cluster ID: 15338ecfc0254b8db6b82c0b66a33a5e
I20260810 00:51:54.959754 17237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:51:54.971390 17237 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:51:54.972595 17237 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:51:54.979784 17237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f: Generated new TSK 0
I20260810 00:51:54.980485 17237 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
I20260810 00:51:55.040617 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:39723
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:35625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:55.355440 17261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:55.355697 17261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:55.355813 17261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:55.362990 17261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:55.363106 17261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:55.363272 17261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:51:55.372548 17261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:35625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:39723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:55.373695 17261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:55.374877 17261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:55.387217 17275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:55.387264 17271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:55.387286 17269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:55.388224 17261 server_base.cc:1034] running on GCE node
I20260810 00:51:55.388756 17261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:55.389552 17261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:55.390726 17261 hybrid_clock.cc:648] HybridClock initialized: now 1786323115390673 us; error 44 us; skew 500 ppm
I20260810 00:51:55.390997 17261 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:55 dist-test-slave-jlzn krb5kdc[17076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323115, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:55.394028 17261 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:51:55.395009 17261 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:55.395095 17261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:55.395346 17261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:55.398022 17261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "b480502420894f279a577708a2877978"
format_stamp: "Formatted at 2026-08-10 00:51:55 on dist-test-slave-jlzn"
server_key: "dd3ef1b0b3e185b6b24f7d3ce47e36c1"
server_key_iv: "caf86b781041382739bc6eabf0360fb3"
server_key_version: "encryptionkey@0"
I20260810 00:51:55.398744 17261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "b480502420894f279a577708a2877978"
format_stamp: "Formatted at 2026-08-10 00:51:55 on dist-test-slave-jlzn"
server_key: "dd3ef1b0b3e185b6b24f7d3ce47e36c1"
server_key_iv: "caf86b781041382739bc6eabf0360fb3"
server_key_version: "encryptionkey@0"
I20260810 00:51:55.403312 17261 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.007s
I20260810 00:51:55.406412 17285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:55.407564 17261 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:51:55.407711 17261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "b480502420894f279a577708a2877978"
format_stamp: "Formatted at 2026-08-10 00:51:55 on dist-test-slave-jlzn"
server_key: "dd3ef1b0b3e185b6b24f7d3ce47e36c1"
server_key_iv: "caf86b781041382739bc6eabf0360fb3"
server_key_version: "encryptionkey@0"
I20260810 00:51:55.407963 17261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:55.431965 17261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:55.435763 17261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:55.436025 17261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:55.436870 17261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:55.438072 17261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:55.438177 17261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:55.438279 17261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:55.438349 17261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:55.466558 17261 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:34305
I20260810 00:51:55.466621 17459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:34305 every 8 connection(s)
I20260810 00:51:55.467324 17261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:51:55.470018 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 17261
I20260810 00:51:55.470222 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:51:55.471076 10629 external_mini_cluster.cc:1428] Setting key f714db9a99cbaf9c98655716ce541ceb
Aug 10 00:51:55 dist-test-slave-jlzn krb5kdc[17076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323115, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:55.486387 17463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:55.469493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:52367 (local address 127.10.97.126:39723)
0810 00:51:55.469927 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:55.469931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:55.470880 (+ 949us) server_negotiation.cc:407] Connection header received
0810 00:51:55.472966 (+ 2086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:55.472970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:55.473030 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:55.473141 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:55.474516 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:55.475510 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:55.477003 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:55.477518 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:55.477652 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:55.480923 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:55.480929 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:55.480933 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:55.480969 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:55.483435 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:55.484297 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:55.484301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:55.484303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:55.484418 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:55.484827 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:55.484830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:55.484832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:55.485080 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:55.485182 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:55.485973 (+ 791us) server_negotiation.cc:299] Negotiation successful
0810 00:51:55.486141 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":240,"threads_started":1}
I20260810 00:51:55.487653 17465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:55.469975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39723 (local address 127.10.97.65:52367)
0810 00:51:55.470725 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:55.470770 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:55.472690 (+ 1920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:55.473321 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:55.473329 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:55.473819 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:55.474346 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:55.474363 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:55.475629 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:55.475633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:55.476866 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:55.476879 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:55.477701 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:55.477706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:55.477884 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:55.478477 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:55.478497 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:55.480706 (+ 2209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:55.483577 (+ 2871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:55.483585 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:55.483598 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:55.484192 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:55.484516 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:55.484520 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:55.484524 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:55.484740 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:55.485199 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:55.485206 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:55.485616 (+ 410us) client_negotiation.cc:769] Sending connection context
0810 00:51:55.485915 (+ 299us) client_negotiation.cc:240] Negotiation successful
0810 00:51:55.486391 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":321,"threads_started":1}
I20260810 00:51:55.489140 17460 heartbeater.cc:347] Connected to a master server at 127.10.97.126:39723
I20260810 00:51:55.489396 17460 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:55.489975 17460 heartbeater.cc:511] Master 127.10.97.126:39723 requested a full tablet report, sending...
I20260810 00:51:55.492164 17149 ts_manager.cc:194] Registered new tserver with Master: b480502420894f279a577708a2877978 (127.10.97.65:34305)
I20260810 00:51:55.494118 17149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:52367
WARNING: no policy specified for kudu/127.10.97.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
I20260810 00:51:55.518327 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:39723
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:35625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:55.827313 17471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:55.827561 17471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:55.827662 17471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:55.834510 17471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:55.834606 17471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:55.834722 17471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:51:55.843676 17471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:35625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:39723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:55.844770 17471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:55.846010 17471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:55.857798 17482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:55.857843 17484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:55.857865 17479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:55.858649 17471 server_base.cc:1034] running on GCE node
I20260810 00:51:55.859208 17471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:55.860103 17471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:55.861296 17471 hybrid_clock.cc:648] HybridClock initialized: now 1786323115861251 us; error 36 us; skew 500 ppm
I20260810 00:51:55.861603 17471 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:55 dist-test-slave-jlzn krb5kdc[17076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323115, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:55.864553 17471 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:51:55.865600 17471 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:55.865693 17471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:55.865988 17471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:55.868783 17471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "11baa1c4c2084004b7fabaa03634e26f"
format_stamp: "Formatted at 2026-08-10 00:51:55 on dist-test-slave-jlzn"
server_key: "016a09da4bf9eb3ddc850b97a7f1aaad"
server_key_iv: "78300d13bb46320bf8daf4d30e363002"
server_key_version: "encryptionkey@0"
I20260810 00:51:55.869554 17471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "11baa1c4c2084004b7fabaa03634e26f"
format_stamp: "Formatted at 2026-08-10 00:51:55 on dist-test-slave-jlzn"
server_key: "016a09da4bf9eb3ddc850b97a7f1aaad"
server_key_iv: "78300d13bb46320bf8daf4d30e363002"
server_key_version: "encryptionkey@0"
I20260810 00:51:55.874272 17471 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:51:55.877501 17490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:55.878625 17471 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:51:55.878770 17471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "11baa1c4c2084004b7fabaa03634e26f"
format_stamp: "Formatted at 2026-08-10 00:51:55 on dist-test-slave-jlzn"
server_key: "016a09da4bf9eb3ddc850b97a7f1aaad"
server_key_iv: "78300d13bb46320bf8daf4d30e363002"
server_key_version: "encryptionkey@0"
I20260810 00:51:55.878960 17471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:55.923323 17471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:55.927070 17471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:55.927311 17471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:55.928232 17471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:55.929483 17471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:55.929569 17471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:55.929680 17471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:55.929752 17471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:55.959285 17471 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:43881
I20260810 00:51:55.959337 17670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:43881 every 8 connection(s)
I20260810 00:51:55.960067 17471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:51:55.965287 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 17471
I20260810 00:51:55.965469 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:51:55.966029 10629 external_mini_cluster.cc:1428] Setting key 2b4023f061d3c117f6af21bd8ddb8087
Aug 10 00:51:55 dist-test-slave-jlzn krb5kdc[17076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323115, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:55.978996 17463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:55.962288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:38177 (local address 127.10.97.126:39723)
0810 00:51:55.962452 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:55.962457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:55.963525 (+ 1068us) server_negotiation.cc:407] Connection header received
0810 00:51:55.965327 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:55.965332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:55.965410 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:55.965507 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:55.967314 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:55.968151 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:55.969628 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:55.970015 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:55.970122 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:55.973670 (+ 3548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:55.973674 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:55.973678 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:55.973709 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:55.976274 (+ 2565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:55.976894 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:55.976899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:55.976902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:55.977062 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:55.977411 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:55.977416 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:55.977420 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:55.977731 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:55.977874 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:55.978462 (+ 588us) server_negotiation.cc:299] Negotiation successful
0810 00:51:55.978670 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":53}
I20260810 00:51:55.979647 17673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:55.962664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39723 (local address 127.10.97.66:38177)
0810 00:51:55.963376 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:55.963411 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:55.965043 (+ 1632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:55.965756 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:55.965767 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:55.966452 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:55.967156 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:55.967176 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:55.968299 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:55.968303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:55.969492 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:55.969504 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:55.970229 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:55.970237 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:55.970438 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:55.971022 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:55.971048 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:55.973466 (+ 2418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:55.976396 (+ 2930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:55.976402 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:55.976412 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:55.976789 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:55.977163 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:55.977166 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:55.977168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:55.977321 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:55.977833 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:55.977838 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:55.978117 (+ 279us) client_negotiation.cc:769] Sending connection context
0810 00:51:55.978342 (+ 225us) client_negotiation.cc:240] Negotiation successful
0810 00:51:55.978669 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":325,"threads_started":1}
I20260810 00:51:55.981066 17671 heartbeater.cc:347] Connected to a master server at 127.10.97.126:39723
I20260810 00:51:55.981329 17671 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:55.981901 17671 heartbeater.cc:511] Master 127.10.97.126:39723 requested a full tablet report, sending...
I20260810 00:51:55.983407 17149 ts_manager.cc:194] Registered new tserver with Master: 11baa1c4c2084004b7fabaa03634e26f (127.10.97.66:43881)
I20260810 00:51:55.984475 17149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:38177
WARNING: no policy specified for kudu/127.10.97.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
I20260810 00:51:56.012320 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:39723
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:35625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:51:56.321449 17677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:56.321722 17677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:56.321856 17677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:56.328742 17677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:56.328847 17677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:56.328996 17677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:51:56.337971 17677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:35625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:39723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:56.339107 17677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:56.340323 17677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:56.352125 17685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:56.352180 17686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:56.352210 17689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:56.353193 17677 server_base.cc:1034] running on GCE node
I20260810 00:51:56.353688 17677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:56.354539 17677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:56.355734 17677 hybrid_clock.cc:648] HybridClock initialized: now 1786323116355681 us; error 41 us; skew 500 ppm
I20260810 00:51:56.356053 17677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:51:56 dist-test-slave-jlzn krb5kdc[17076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323116, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:51:56.358865 17677 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:51:56.359939 17677 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:56.360052 17677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:56.360332 17677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:56.362960 17677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "7ce67cc21b564befb24363c858a450ac"
format_stamp: "Formatted at 2026-08-10 00:51:56 on dist-test-slave-jlzn"
server_key: "db7a299a0e423469ba2b15658381ba79"
server_key_iv: "d3d16e41ec67e3a97a3ade413e6c1838"
server_key_version: "encryptionkey@0"
I20260810 00:51:56.363730 17677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "7ce67cc21b564befb24363c858a450ac"
format_stamp: "Formatted at 2026-08-10 00:51:56 on dist-test-slave-jlzn"
server_key: "db7a299a0e423469ba2b15658381ba79"
server_key_iv: "d3d16e41ec67e3a97a3ade413e6c1838"
server_key_version: "encryptionkey@0"
I20260810 00:51:56.368347 17677 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 00:51:56.371415 17699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:56.372618 17677 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:51:56.372764 17677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "7ce67cc21b564befb24363c858a450ac"
format_stamp: "Formatted at 2026-08-10 00:51:56 on dist-test-slave-jlzn"
server_key: "db7a299a0e423469ba2b15658381ba79"
server_key_iv: "d3d16e41ec67e3a97a3ade413e6c1838"
server_key_version: "encryptionkey@0"
I20260810 00:51:56.372964 17677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:56.395848 17677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:56.399446 17677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:56.399693 17677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:56.400555 17677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:56.401741 17677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:56.401832 17677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:56.401933 17677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:56.402004 17677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:56.429927 17677 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:41641
I20260810 00:51:56.429986 17875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:41641 every 8 connection(s)
I20260810 00:51:56.430684 17677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:51:56.439086 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 17677
I20260810 00:51:56.439237 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:51:56.439647 10629 external_mini_cluster.cc:1428] Setting key f15003b024681e4390013f4fa9ab9053
Aug 10 00:51:56 dist-test-slave-jlzn krb5kdc[17076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323116, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:51:56.448175 17463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:56.432875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:60443 (local address 127.10.97.126:39723)
0810 00:51:56.433036 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:56.433040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:56.434045 (+ 1005us) server_negotiation.cc:407] Connection header received
0810 00:51:56.435719 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:56.435723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:56.435790 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:56.435898 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:56.437210 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.437958 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.439453 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.439946 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.440061 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:56.443378 (+ 3317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:56.443383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:56.443387 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:56.443418 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:56.445770 (+ 2352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:56.446419 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:56.446423 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:56.446425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:56.446538 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:56.446859 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:56.446862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:56.446864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:56.447122 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:56.447226 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:56.447739 (+ 513us) server_negotiation.cc:299] Negotiation successful
0810 00:51:56.447924 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 00:51:56.449007 17879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:56.433252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39723 (local address 127.10.97.67:60443)
0810 00:51:56.433904 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:56.433939 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:56.435467 (+ 1528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:56.436077 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:56.436084 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:56.436549 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:56.437081 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:56.437098 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.438070 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.438073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:56.439326 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:56.439339 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.440201 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.440208 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:56.440408 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:56.440997 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:51:56.441018 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:56.443194 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:56.445878 (+ 2684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:56.445884 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:56.445912 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:56.446339 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:56.446617 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:51:56.446620 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:51:56.446622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:51:56.446781 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:51:56.447216 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:56.447221 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:56.447481 (+ 260us) client_negotiation.cc:769] Sending connection context
0810 00:51:56.447682 (+ 201us) client_negotiation.cc:240] Negotiation successful
0810 00:51:56.448050 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":269,"threads_started":1}
I20260810 00:51:56.450157 17877 heartbeater.cc:347] Connected to a master server at 127.10.97.126:39723
I20260810 00:51:56.450408 17877 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:56.450964 17877 heartbeater.cc:511] Master 127.10.97.126:39723 requested a full tablet report, sending...
I20260810 00:51:56.452250 17149 ts_manager.cc:194] Registered new tserver with Master: 7ce67cc21b564befb24363c858a450ac (127.10.97.67:41641)
I20260810 00:51:56.453209 17149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:60443
I20260810 00:51:56.454581 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:51:56.470178 17463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:56.459061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53818 (local address 127.10.97.126:39723)
0810 00:51:56.459209 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:56.459213 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:56.459667 (+ 454us) server_negotiation.cc:407] Connection header received
0810 00:51:56.459778 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:56.459782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:56.459877 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:56.459949 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:56.461214 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.461888 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.463060 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.463485 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.463568 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:56.465508 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:56.465513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:56.465516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:56.465544 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:56.467783 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:56.468515 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:56.468518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:56.468520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:56.468634 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:56.469019 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:56.469025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:56.469028 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:56.469305 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:56.469475 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:56.469820 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 00:51:56.469962 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 00:51:56.473483 17149 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:51:56.480132 17149 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:51:56.497041 17460 heartbeater.cc:503] Master 127.10.97.126:39723 was elected leader, sending a full tablet report...
I20260810 00:51:56.512913 17898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:56.491460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:41641 (local address 127.0.0.1:41172)
0810 00:51:56.505640 (+ 14180us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:56.505662 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:56.505772 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:56.506286 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:56.506290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:56.506314 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:56.506679 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:56.506691 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.508533 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.508539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:56.510152 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:56.510170 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.512003 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.512010 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:56.512189 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:56.512238 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 00:51:56.512377 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 00:51:56.512548 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14050,"thread_start_us":11257,"threads_started":1}
I20260810 00:51:56.513204 17910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:56.491424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42296 (local address 127.10.97.66:43881)
0810 00:51:56.501122 (+ 9698us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:56.501132 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:56.505093 (+ 3961us) server_negotiation.cc:407] Connection header received
0810 00:51:56.505197 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:56.505205 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:56.505452 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:56.505646 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:56.506385 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.507973 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.510595 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.512178 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.512307 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:56.512480 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:56.512730 (+ 250us) server_negotiation.cc:299] Negotiation successful
0810 00:51:56.512883 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9529,"thread_start_us":9424,"threads_started":1}
I20260810 00:51:56.513506 17911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:56.492078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41172 (local address 127.10.97.67:41641)
0810 00:51:56.501618 (+ 9540us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:56.501626 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:56.505761 (+ 4135us) server_negotiation.cc:407] Connection header received
0810 00:51:56.505896 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:56.505903 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:56.506102 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:56.506281 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:56.506904 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.508407 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.510305 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.511702 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.511852 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:56.512072 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:56.513019 (+ 947us) server_negotiation.cc:299] Negotiation successful
0810 00:51:56.513189 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9379,"thread_start_us":9214,"threads_started":1}
I20260810 00:51:56.513702 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:56.490819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34305 (local address 127.0.0.1:53434)
0810 00:51:56.503098 (+ 12279us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:56.503150 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:56.503312 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:56.506689 (+ 3377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:56.506694 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:56.506722 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:56.507069 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:56.507079 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.508824 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.508830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:56.510817 (+ 1987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:56.510830 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.512921 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.512929 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:56.513087 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:56.513116 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 00:51:56.513263 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 00:51:56.513391 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12032,"thread_start_us":13767,"threads_started":1}
I20260810 00:51:56.514025 17895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:56.491227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:43881 (local address 127.0.0.1:42296)
0810 00:51:56.504836 (+ 13609us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:56.504857 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:56.504976 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:56.505918 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:56.505923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:56.505945 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:56.506196 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:56.506203 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.508139 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.508145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:56.510445 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:56.510460 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.512380 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.512387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:56.512560 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:56.512586 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 00:51:56.512663 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 00:51:56.512779 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13497,"mutex_wait_us":1,"thread_start_us":14181,"threads_started":1}
I20260810 00:51:56.514158 17897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:56.491067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53434 (local address 127.10.97.65:34305)
0810 00:51:56.506185 (+ 15118us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:56.506193 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:56.506218 (+ 25us) server_negotiation.cc:407] Connection header received
0810 00:51:56.506300 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:56.506306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:56.506502 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:56.506671 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:56.507221 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.508669 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.510982 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.512439 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.512556 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:56.512755 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:56.513695 (+ 940us) server_negotiation.cc:299] Negotiation successful
0810 00:51:56.513853 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14941,"thread_start_us":15758,"threads_started":1}
I20260810 00:51:56.517675 17780 tablet_service.cc:1467] Processing CreateTablet for tablet 84eeee1aa8474b069cc78866f5525230 (DEFAULT_TABLE table=test-table [id=3150e72ebcc741aa9c91a1712599168c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:56.517972 17348 tablet_service.cc:1467] Processing CreateTablet for tablet 84eeee1aa8474b069cc78866f5525230 (DEFAULT_TABLE table=test-table [id=3150e72ebcc741aa9c91a1712599168c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:56.518361 17569 tablet_service.cc:1467] Processing CreateTablet for tablet 84eeee1aa8474b069cc78866f5525230 (DEFAULT_TABLE table=test-table [id=3150e72ebcc741aa9c91a1712599168c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:56.519070 17780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84eeee1aa8474b069cc78866f5525230. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:56.519335 17348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84eeee1aa8474b069cc78866f5525230. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:56.519609 17569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84eeee1aa8474b069cc78866f5525230. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:56.529989 17912 tablet_bootstrap.cc:492] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac: Bootstrap starting.
I20260810 00:51:56.532912 17913 tablet_bootstrap.cc:492] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978: Bootstrap starting.
I20260810 00:51:56.533092 17912 tablet_bootstrap.cc:654] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:56.534597 17912 log.cc:824] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:56.536288 17913 tablet_bootstrap.cc:654] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:56.537525 17913 log.cc:824] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:56.537787 17914 tablet_bootstrap.cc:492] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f: Bootstrap starting.
I20260810 00:51:56.538425 17912 tablet_bootstrap.cc:492] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac: No bootstrap required, opened a new log
I20260810 00:51:56.538656 17912 ts_tablet_manager.cc:1397] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260810 00:51:56.540453 17913 tablet_bootstrap.cc:492] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978: No bootstrap required, opened a new log
I20260810 00:51:56.540514 17914 tablet_bootstrap.cc:654] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:56.540704 17913 ts_tablet_manager.cc:1397] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260810 00:51:56.541495 17914 log.cc:824] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:56.543841 17914 tablet_bootstrap.cc:492] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f: No bootstrap required, opened a new log
I20260810 00:51:56.544149 17914 ts_tablet_manager.cc:1397] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260810 00:51:56.549176 17912 raft_consensus.cc:348] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b480502420894f279a577708a2877978" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34305 } } peers { permanent_uuid: "11baa1c4c2084004b7fabaa03634e26f" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 43881 } } peers { permanent_uuid: "7ce67cc21b564befb24363c858a450ac" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 41641 } }
I20260810 00:51:56.549582 17912 raft_consensus.cc:374] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:56.549688 17912 raft_consensus.cc:729] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ce67cc21b564befb24363c858a450ac, State: Initialized, Role: FOLLOWER
I20260810 00:51:56.550271 17912 consensus_queue.cc:260] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b480502420894f279a577708a2877978" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34305 } } peers { permanent_uuid: "11baa1c4c2084004b7fabaa03634e26f" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 43881 } } peers { permanent_uuid: "7ce67cc21b564befb24363c858a450ac" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 41641 } }
I20260810 00:51:56.550518 17913 raft_consensus.cc:348] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b480502420894f279a577708a2877978" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34305 } } peers { permanent_uuid: "11baa1c4c2084004b7fabaa03634e26f" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 43881 } } peers { permanent_uuid: "7ce67cc21b564befb24363c858a450ac" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 41641 } }
I20260810 00:51:56.550978 17913 raft_consensus.cc:374] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:56.551121 17913 raft_consensus.cc:729] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b480502420894f279a577708a2877978, State: Initialized, Role: FOLLOWER
I20260810 00:51:56.551342 17877 heartbeater.cc:503] Master 127.10.97.126:39723 was elected leader, sending a full tablet report...
I20260810 00:51:56.551730 17914 raft_consensus.cc:348] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b480502420894f279a577708a2877978" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34305 } } peers { permanent_uuid: "11baa1c4c2084004b7fabaa03634e26f" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 43881 } } peers { permanent_uuid: "7ce67cc21b564befb24363c858a450ac" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 41641 } }
I20260810 00:51:56.552040 17912 ts_tablet_manager.cc:1428] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260810 00:51:56.551955 17913 consensus_queue.cc:260] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b480502420894f279a577708a2877978" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34305 } } peers { permanent_uuid: "11baa1c4c2084004b7fabaa03634e26f" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 43881 } } peers { permanent_uuid: "7ce67cc21b564befb24363c858a450ac" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 41641 } }
I20260810 00:51:56.552277 17914 raft_consensus.cc:374] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:56.552381 17914 raft_consensus.cc:729] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11baa1c4c2084004b7fabaa03634e26f, State: Initialized, Role: FOLLOWER
I20260810 00:51:56.553130 17914 consensus_queue.cc:260] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b480502420894f279a577708a2877978" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34305 } } peers { permanent_uuid: "11baa1c4c2084004b7fabaa03634e26f" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 43881 } } peers { permanent_uuid: "7ce67cc21b564befb24363c858a450ac" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 41641 } }
I20260810 00:51:56.553785 17913 ts_tablet_manager.cc:1428] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260810 00:51:56.554605 17671 heartbeater.cc:503] Master 127.10.97.126:39723 was elected leader, sending a full tablet report...
I20260810 00:51:56.557036 17914 ts_tablet_manager.cc:1428] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f: Time spent starting tablet: real 0.013s user 0.007s sys 0.003s
I20260810 00:51:56.625211 17926 raft_consensus.cc:482] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:51:56.625519 17926 raft_consensus.cc:504] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b480502420894f279a577708a2877978" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34305 } } peers { permanent_uuid: "11baa1c4c2084004b7fabaa03634e26f" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 43881 } } peers { permanent_uuid: "7ce67cc21b564befb24363c858a450ac" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 41641 } }
I20260810 00:51:56.627064 17926 leader_election.cc:288] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11baa1c4c2084004b7fabaa03634e26f (127.10.97.66:43881), 7ce67cc21b564befb24363c858a450ac (127.10.97.67:41641)
I20260810 00:51:56.633836 17910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:56.627364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:48783 (local address 127.10.97.66:43881)
0810 00:51:56.627555 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:56.627562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:56.628676 (+ 1114us) server_negotiation.cc:407] Connection header received
0810 00:51:56.628828 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:56.628832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:56.628888 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:56.628962 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:56.629539 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.630229 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.632168 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.632924 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.633057 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:56.633130 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:56.633465 (+ 335us) server_negotiation.cc:299] Negotiation successful
0810 00:51:56.633588 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 00:51:56.633836 17911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:56.627619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:47297 (local address 127.10.97.67:41641)
0810 00:51:56.627854 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:56.627860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:56.628575 (+ 715us) server_negotiation.cc:407] Connection header received
0810 00:51:56.628759 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:56.628765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:56.628823 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:56.628896 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:51:56.629481 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.630220 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.632194 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.632931 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.633071 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:56.633153 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:56.633482 (+ 329us) server_negotiation.cc:299] Negotiation successful
0810 00:51:56.633602 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 00:51:56.633984 17935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:56.627469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:43881 (local address 127.10.97.65:48783)
0810 00:51:56.628565 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:56.628584 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:56.628702 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:56.628987 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:56.628991 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:56.629019 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:56.629383 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:56.629392 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.630355 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.630361 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:56.632022 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:56.632035 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.633151 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.633158 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:56.633289 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:56.633338 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 00:51:56.633417 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 00:51:56.633532 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"mutex_wait_us":77,"thread_start_us":795,"threads_started":1}
I20260810 00:51:56.634023 17931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:56.627409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:41641 (local address 127.10.97.65:47297)
0810 00:51:56.628475 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:56.628491 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:56.628624 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:56.628921 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:56.628925 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:56.628950 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:51:56.629338 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:56.629351 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.630339 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.630344 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:56.632039 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:51:56.632050 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.633190 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.633197 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:51:56.633337 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:56.633359 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 00:51:56.633434 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 00:51:56.633547 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"mutex_wait_us":76,"spinlock_wait_cycles":1792,"thread_start_us":629,"threads_started":1}
I20260810 00:51:56.634874 17810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84eeee1aa8474b069cc78866f5525230" candidate_uuid: "b480502420894f279a577708a2877978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ce67cc21b564befb24363c858a450ac" is_pre_election: true
I20260810 00:51:56.635027 17609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84eeee1aa8474b069cc78866f5525230" candidate_uuid: "b480502420894f279a577708a2877978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11baa1c4c2084004b7fabaa03634e26f" is_pre_election: true
I20260810 00:51:56.635331 17810 raft_consensus.cc:2454] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b480502420894f279a577708a2877978 in term 0.
I20260810 00:51:56.635452 17609 raft_consensus.cc:2454] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b480502420894f279a577708a2877978 in term 0.
I20260810 00:51:56.635924 17292 leader_election.cc:302] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ce67cc21b564befb24363c858a450ac, b480502420894f279a577708a2877978; no voters:
I20260810 00:51:56.636288 17926 raft_consensus.cc:2781] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:51:56.636411 17926 raft_consensus.cc:482] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:51:56.636507 17926 raft_consensus.cc:3037] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:56.638585 17926 raft_consensus.cc:504] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b480502420894f279a577708a2877978" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34305 } } peers { permanent_uuid: "11baa1c4c2084004b7fabaa03634e26f" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 43881 } } peers { permanent_uuid: "7ce67cc21b564befb24363c858a450ac" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 41641 } }
I20260810 00:51:56.639149 17926 leader_election.cc:288] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [CANDIDATE]: Term 1 election: Requested vote from peers 11baa1c4c2084004b7fabaa03634e26f (127.10.97.66:43881), 7ce67cc21b564befb24363c858a450ac (127.10.97.67:41641)
I20260810 00:51:56.639487 17609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84eeee1aa8474b069cc78866f5525230" candidate_uuid: "b480502420894f279a577708a2877978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11baa1c4c2084004b7fabaa03634e26f"
I20260810 00:51:56.639647 17609 raft_consensus.cc:3037] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:56.639595 17810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84eeee1aa8474b069cc78866f5525230" candidate_uuid: "b480502420894f279a577708a2877978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ce67cc21b564befb24363c858a450ac"
I20260810 00:51:56.639786 17810 raft_consensus.cc:3037] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:56.641947 17609 raft_consensus.cc:2454] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b480502420894f279a577708a2877978 in term 1.
I20260810 00:51:56.642028 17810 raft_consensus.cc:2454] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b480502420894f279a577708a2877978 in term 1.
I20260810 00:51:56.642334 17291 leader_election.cc:302] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11baa1c4c2084004b7fabaa03634e26f, b480502420894f279a577708a2877978; no voters:
I20260810 00:51:56.642597 17926 raft_consensus.cc:2781] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:56.642982 17926 raft_consensus.cc:686] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [term 1 LEADER]: Becoming Leader. State: Replica: b480502420894f279a577708a2877978, State: Running, Role: LEADER
I20260810 00:51:56.643453 17926 consensus_queue.cc:237] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b480502420894f279a577708a2877978" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34305 } } peers { permanent_uuid: "11baa1c4c2084004b7fabaa03634e26f" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 43881 } } peers { permanent_uuid: "7ce67cc21b564befb24363c858a450ac" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 41641 } }
I20260810 00:51:56.647395 17148 catalog_manager.cc:5495] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 reported cstate change: term changed from 0 to 1, leader changed from <none> to b480502420894f279a577708a2877978 (127.10.97.65). New cstate: current_term: 1 leader_uuid: "b480502420894f279a577708a2877978" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b480502420894f279a577708a2877978" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11baa1c4c2084004b7fabaa03634e26f" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 43881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ce67cc21b564befb24363c858a450ac" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 41641 } health_report { overall_health: UNKNOWN } } }
I20260810 00:51:56.665693 17897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:56.660737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53436 (local address 127.10.97.65:34305)
0810 00:51:56.660887 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:56.660891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:56.660991 (+ 100us) server_negotiation.cc:407] Connection header received
0810 00:51:56.661182 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:56.661186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:56.661264 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:56.661384 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:51:56.661747 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.662424 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.663731 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.664176 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.664281 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:56.664718 (+ 437us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:51:56.665158 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:51:56.665282 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:56.665389 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 00:51:56.665464 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 00:51:56.672994 17609 raft_consensus.cc:1261] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f [term 1 FOLLOWER]: Refusing update from remote peer b480502420894f279a577708a2877978: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:51:56.672995 17810 raft_consensus.cc:1261] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac [term 1 FOLLOWER]: Refusing update from remote peer b480502420894f279a577708a2877978: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:51:56.673951 17937 consensus_queue.cc:1035] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ce67cc21b564befb24363c858a450ac" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 41641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:51:56.674274 17926 consensus_queue.cc:1035] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11baa1c4c2084004b7fabaa03634e26f" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 43881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 00:51:56.681981 17878 tablet.cc:2369] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:51:56.698118 17148 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:51:56.698348 17148 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:51:56.701663 17148 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a169dccfd7d14aa091a1296e6c8cbe1f: Sending DeleteTablet for 3 replicas of tablet 84eeee1aa8474b069cc78866f5525230
I20260810 00:51:56.702666 17780 tablet_service.cc:1514] Processing DeleteTablet for tablet 84eeee1aa8474b069cc78866f5525230 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:56 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:41172
I20260810 00:51:56.702683 17348 tablet_service.cc:1514] Processing DeleteTablet for tablet 84eeee1aa8474b069cc78866f5525230 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:56 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:53434
I20260810 00:51:56.703301 17569 tablet_service.cc:1514] Processing DeleteTablet for tablet 84eeee1aa8474b069cc78866f5525230 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:56 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:42296
I20260810 00:51:56.703928 17958 tablet_replica.cc:330] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978: stopping tablet replica
I20260810 00:51:56.704317 17958 raft_consensus.cc:2229] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:51:56.704865 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 17261
I20260810 00:51:56.704921 17958 raft_consensus.cc:2258] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:56.705636 17955 tablet_replica.cc:330] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac: stopping tablet replica
I20260810 00:51:56.705987 17955 raft_consensus.cc:2229] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:51:56.706408 17955 raft_consensus.cc:2258] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:56.708334 17958 ts_tablet_manager.cc:1905] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:56.715976 17955 ts_tablet_manager.cc:1905] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:56.717828 17959 tablet_replica.cc:330] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f: stopping tablet replica
I20260810 00:51:56.718097 17958 ts_tablet_manager.cc:1918] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:51:56.718205 17959 raft_consensus.cc:2229] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:51:56.718281 17958 log.cc:1196] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/84eeee1aa8474b069cc78866f5525230
I20260810 00:51:56.718721 17959 raft_consensus.cc:2258] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:56.719676 17958 ts_tablet_manager.cc:1939] T 84eeee1aa8474b069cc78866f5525230 P b480502420894f279a577708a2877978: Deleting consensus metadata
I20260810 00:51:56.721561 17129 catalog_manager.cc:4841] TS b480502420894f279a577708a2877978 (127.10.97.65:34305): tablet 84eeee1aa8474b069cc78866f5525230 (table test-table [id=3150e72ebcc741aa9c91a1712599168c]) successfully deleted
I20260810 00:51:56.722697 17955 ts_tablet_manager.cc:1918] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:51:56.722836 17955 log.cc:1196] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/84eeee1aa8474b069cc78866f5525230
I20260810 00:51:56.723299 17955 ts_tablet_manager.cc:1939] T 84eeee1aa8474b069cc78866f5525230 P 7ce67cc21b564befb24363c858a450ac: Deleting consensus metadata
I20260810 00:51:56.724860 17959 ts_tablet_manager.cc:1905] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:56.724905 17127 catalog_manager.cc:4841] TS 7ce67cc21b564befb24363c858a450ac (127.10.97.67:41641): tablet 84eeee1aa8474b069cc78866f5525230 (table test-table [id=3150e72ebcc741aa9c91a1712599168c]) successfully deleted
Aug 10 00:51:56 dist-test-slave-jlzn krb5kdc[17076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:51:56.731549 17959 ts_tablet_manager.cc:1918] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:51:56.731734 17959 log.cc:1196] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/84eeee1aa8474b069cc78866f5525230
I20260810 00:51:56.732287 17959 ts_tablet_manager.cc:1939] T 84eeee1aa8474b069cc78866f5525230 P 11baa1c4c2084004b7fabaa03634e26f: Deleting consensus metadata
I20260810 00:51:56.733639 17897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:56.706147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53452 (local address 127.10.97.65:34305)
0810 00:51:56.706298 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:56.706304 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:56.714937 (+ 8633us) server_negotiation.cc:407] Connection header received
0810 00:51:56.715147 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:56.715152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:56.715230 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:56.715329 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:56.719981 (+ 4652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.721270 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.723211 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.723913 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.724040 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:56.728061 (+ 4021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:56.728071 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:56.728083 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:56.728117 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:56.730823 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:56.731437 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:56.731444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:56.731446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:56.731608 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:56.732037 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:56.732043 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:56.732046 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:56.732406 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:56.732707 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:56.732968 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 00:51:56.733184 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 00:51:56.733922 17126 catalog_manager.cc:4841] TS 11baa1c4c2084004b7fabaa03634e26f (127.10.97.66:43881): tablet 84eeee1aa8474b069cc78866f5525230 (table test-table [id=3150e72ebcc741aa9c91a1712599168c]) successfully deleted
I20260810 00:51:56.734290 17453 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:56.828285 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 17261
I20260810 00:51:56.849514 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 17471
Aug 10 00:51:56 dist-test-slave-jlzn krb5kdc[17076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.66@KRBTEST.COM
I20260810 00:51:56.862929 17910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:56.850202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42304 (local address 127.10.97.66:43881)
0810 00:51:56.850358 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:56.850362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:56.850382 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:51:56.850529 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:56.850535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:56.850602 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:56.850683 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:56.851839 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.852602 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.853753 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.854289 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.854383 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:56.858300 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:56.858312 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:56.858328 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:56.858360 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:56.860833 (+ 2473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:56.861405 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:56.861410 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:56.861413 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:56.861536 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:56.861805 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:56.861808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:56.861810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:56.862160 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:56.862378 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:56.862541 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 00:51:56.862693 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 00:51:56.863554 17664 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:56.952411 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 17471
I20260810 00:51:56.972806 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 17677
Aug 10 00:51:56 dist-test-slave-jlzn krb5kdc[17076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.67@KRBTEST.COM
I20260810 00:51:56.986311 17911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:56.973501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41178 (local address 127.10.97.67:41641)
0810 00:51:56.973653 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:56.973657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:56.973675 (+ 18us) server_negotiation.cc:407] Connection header received
0810 00:51:56.973804 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:56.973807 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:56.973878 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:56.973960 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:56.975125 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.975931 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.977211 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:51:56.977622 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:51:56.977708 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:51:56.981738 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:56.981753 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:56.981771 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:51:56.981801 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:56.984235 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:56.984766 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:56.984772 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:56.984775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:56.984898 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:51:56.985167 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:51:56.985170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:51:56.985172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:51:56.985495 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:56.985699 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:56.985913 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 00:51:56.986078 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 00:51:56.986913 17869 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:57.071924 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 17677
I20260810 00:51:57.092767 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 17100
I20260810 00:51:57.093801 17199 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:57.199025 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 17100
2026-08-10T00:51:57Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4082 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T00:51:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:51:57Z Disabled control of system clock
I20260810 00:51:57.271087 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:33729
--webserver_interface=127.10.97.126
--webserver_port=0
--builtin_ntp_servers=127.10.97.84:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:33729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 00:51:57.576503 17973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:57.576766 17973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:57.576853 17973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:57.583567 17973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:51:57.583665 17973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:57.583709 17973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:57.583762 17973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:51:57.583823 17973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:51:57.592409 17973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:33729
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:33729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:57.593433 17973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:57.594993 17973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:57.605804 17980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:57.605862 17981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:57.605886 17983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:57.606658 17973 server_base.cc:1034] running on GCE node
I20260810 00:51:57.607399 17973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:57.608713 17973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:57.609897 17973 hybrid_clock.cc:648] HybridClock initialized: now 1786323117609851 us; error 42 us; skew 500 ppm
I20260810 00:51:57.610210 17973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:51:57.615347 17973 webserver.cc:460] Webserver started at http://127.10.97.126:43329/ using document root <none> and password file <none>
I20260810 00:51:57.616065 17973 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:57.616158 17973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:57.616474 17973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:57.619227 17973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "f02f2b5a46034a7dbf90dbd6a67ebe51"
format_stamp: "Formatted at 2026-08-10 00:51:57 on dist-test-slave-jlzn"
server_key: "5986c9733c7f4025ae2c4140de4bca07"
server_key_iv: "e558b8566dc7a42411c013fdb4a93231"
server_key_version: "encryptionkey@0"
I20260810 00:51:57.620035 17973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "f02f2b5a46034a7dbf90dbd6a67ebe51"
format_stamp: "Formatted at 2026-08-10 00:51:57 on dist-test-slave-jlzn"
server_key: "5986c9733c7f4025ae2c4140de4bca07"
server_key_iv: "e558b8566dc7a42411c013fdb4a93231"
server_key_version: "encryptionkey@0"
I20260810 00:51:57.624720 17973 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:51:57.627785 17989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:57.629002 17973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:51:57.629173 17973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "f02f2b5a46034a7dbf90dbd6a67ebe51"
format_stamp: "Formatted at 2026-08-10 00:51:57 on dist-test-slave-jlzn"
server_key: "5986c9733c7f4025ae2c4140de4bca07"
server_key_iv: "e558b8566dc7a42411c013fdb4a93231"
server_key_version: "encryptionkey@0"
I20260810 00:51:57.629366 17973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:57.634702 17973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:57.635365 17973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:57.635637 17973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:57.652055 17973 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:33729
I20260810 00:51:57.652086 18075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:33729 every 8 connection(s)
I20260810 00:51:57.653196 17973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:51:57.656457 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 17973
I20260810 00:51:57.656559 18076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:57.656663 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:51:57.657274 10629 external_mini_cluster.cc:1428] Setting key 73ace35916556a0f84066b6af461e02d
I20260810 00:51:57.665149 18079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:57.659064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51946 (local address 127.10.97.126:33729)
0810 00:51:57.660054 (+ 990us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:57.660071 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:57.660115 (+ 44us) server_negotiation.cc:407] Connection header received
0810 00:51:57.661961 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:57.661981 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:57.662361 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:57.662673 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:57.663695 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:57.663701 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:57.663713 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:51:57.663727 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:57.663789 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:51:57.663838 (+ 49us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:57.664014 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:57.664105 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 00:51:57.664308 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":360,"threads_started":1}
I20260810 00:51:57.668551 18076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51: Bootstrap starting.
I20260810 00:51:57.671298 18076 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:57.672272 18076 log.cc:824] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:57.674554 18076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51: No bootstrap required, opened a new log
I20260810 00:51:57.681316 18076 raft_consensus.cc:348] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f02f2b5a46034a7dbf90dbd6a67ebe51" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33729 } }
I20260810 00:51:57.681613 18076 raft_consensus.cc:374] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:57.681707 18076 raft_consensus.cc:729] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f02f2b5a46034a7dbf90dbd6a67ebe51, State: Initialized, Role: FOLLOWER
I20260810 00:51:57.682178 18076 consensus_queue.cc:260] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f02f2b5a46034a7dbf90dbd6a67ebe51" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33729 } }
I20260810 00:51:57.682384 18076 raft_consensus.cc:388] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:51:57.682499 18076 raft_consensus.cc:482] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:51:57.682647 18076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:57.684777 18076 raft_consensus.cc:504] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f02f2b5a46034a7dbf90dbd6a67ebe51" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33729 } }
I20260810 00:51:57.685205 18076 leader_election.cc:302] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f02f2b5a46034a7dbf90dbd6a67ebe51; no voters:
I20260810 00:51:57.685706 18076 leader_election.cc:288] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:51:57.685811 18082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:51:57.686417 18082 raft_consensus.cc:686] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51 [term 1 LEADER]: Becoming Leader. State: Replica: f02f2b5a46034a7dbf90dbd6a67ebe51, State: Running, Role: LEADER
I20260810 00:51:57.686950 18082 consensus_queue.cc:237] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f02f2b5a46034a7dbf90dbd6a67ebe51" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33729 } }
I20260810 00:51:57.687464 18076 sys_catalog.cc:564] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:51:57.689312 18083 sys_catalog.cc:455] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f02f2b5a46034a7dbf90dbd6a67ebe51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f02f2b5a46034a7dbf90dbd6a67ebe51" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33729 } } }
I20260810 00:51:57.689795 18083 sys_catalog.cc:458] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51 [sys.catalog]: This master's current role is: LEADER
I20260810 00:51:57.689334 18084 sys_catalog.cc:455] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f02f2b5a46034a7dbf90dbd6a67ebe51. Latest consensus state: current_term: 1 leader_uuid: "f02f2b5a46034a7dbf90dbd6a67ebe51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f02f2b5a46034a7dbf90dbd6a67ebe51" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33729 } } }
I20260810 00:51:57.690127 18084 sys_catalog.cc:458] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51 [sys.catalog]: This master's current role is: LEADER
I20260810 00:51:57.691347 18090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:51:57.696267 18090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:51:57.704102 18090 catalog_manager.cc:1347] Generated new cluster ID: d8200765d5ec4343b9dd8249ec3f3581
I20260810 00:51:57.704196 18090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:51:57.735328 18090 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:51:57.735688 18090 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:51:57.745748 18090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51: Generated new TSK 0
I20260810 00:51:57.746397 18090 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:51:57.761809 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:33729
--builtin_ntp_servers=127.10.97.84:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 00:51:58.079892 18107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:58.080139 18107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:58.080260 18107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:58.087148 18107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:58.087242 18107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:58.087393 18107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:51:58.096072 18107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:58.097144 18107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:58.098781 18107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:58.110735 18116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:58.110733 18118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:58.110836 18115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:58.111744 18107 server_base.cc:1034] running on GCE node
I20260810 00:51:58.112334 18107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:58.113135 18107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:58.114322 18107 hybrid_clock.cc:648] HybridClock initialized: now 1786323118114274 us; error 46 us; skew 500 ppm
I20260810 00:51:58.114596 18107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:51:58.116667 18107 webserver.cc:460] Webserver started at http://127.10.97.65:42271/ using document root <none> and password file <none>
I20260810 00:51:58.117269 18107 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:58.117362 18107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:58.117669 18107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:58.120421 18107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "b552fbcf05734ef1a00e65e3dc690bd4"
format_stamp: "Formatted at 2026-08-10 00:51:58 on dist-test-slave-jlzn"
server_key: "c3f14cc0b72569813d745d6461cc1c0a"
server_key_iv: "043357a9e133e53d20bcf85b675fcb4b"
server_key_version: "encryptionkey@0"
I20260810 00:51:58.121147 18107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "b552fbcf05734ef1a00e65e3dc690bd4"
format_stamp: "Formatted at 2026-08-10 00:51:58 on dist-test-slave-jlzn"
server_key: "c3f14cc0b72569813d745d6461cc1c0a"
server_key_iv: "043357a9e133e53d20bcf85b675fcb4b"
server_key_version: "encryptionkey@0"
I20260810 00:51:58.125726 18107 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 00:51:58.128801 18125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:58.129962 18107 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:51:58.130119 18107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "b552fbcf05734ef1a00e65e3dc690bd4"
format_stamp: "Formatted at 2026-08-10 00:51:58 on dist-test-slave-jlzn"
server_key: "c3f14cc0b72569813d745d6461cc1c0a"
server_key_iv: "043357a9e133e53d20bcf85b675fcb4b"
server_key_version: "encryptionkey@0"
I20260810 00:51:58.130324 18107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:58.135761 18107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:58.136502 18107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:58.136737 18107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:58.137563 18107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:58.138916 18107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:58.139019 18107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:58.139119 18107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:58.139194 18107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:58.166304 18107 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:40977
I20260810 00:51:58.166353 18296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:40977 every 8 connection(s)
I20260810 00:51:58.167511 18107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:51:58.173540 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 18107
I20260810 00:51:58.173732 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:51:58.174427 10629 external_mini_cluster.cc:1428] Setting key e9db66ea9d0f43ab175e774e4be63620
I20260810 00:51:58.175468 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:58.169948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:60781 (local address 127.10.97.126:33729)
0810 00:51:58.170375 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:58.170381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:58.171537 (+ 1156us) server_negotiation.cc:407] Connection header received
0810 00:51:58.173193 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:58.173199 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:58.173270 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:58.173385 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:58.174718 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:58.174723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:58.174728 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:51:58.174748 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:58.174800 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:51:58.174816 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:58.174941 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:58.175101 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 00:51:58.175167 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":190,"threads_started":1}
I20260810 00:51:58.176311 18300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:58.170526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33729 (local address 127.10.97.65:60781)
0810 00:51:58.171383 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:58.171425 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:58.172923 (+ 1498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:58.173622 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:58.173630 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:58.173748 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:58.174419 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:51:58.174422 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:58.174458 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:51:58.174545 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:58.174907 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:58.174912 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:58.174917 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 00:51:58.175146 (+ 229us) client_negotiation.cc:240] Negotiation successful
0810 00:51:58.175378 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":250,"threads_started":1}
I20260810 00:51:58.177264 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:33729
--builtin_ntp_servers=127.10.97.84:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 00:51:58.177703 18297 heartbeater.cc:347] Connected to a master server at 127.10.97.126:33729
I20260810 00:51:58.177947 18297 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:58.178467 18297 heartbeater.cc:511] Master 127.10.97.126:33729 requested a full tablet report, sending...
I20260810 00:51:58.180552 18013 ts_manager.cc:194] Registered new tserver with Master: b552fbcf05734ef1a00e65e3dc690bd4 (127.10.97.65:40977)
W20260810 00:51:58.488634 18301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:58.488881 18301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:58.488989 18301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:58.495875 18301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:58.495967 18301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:58.496073 18301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:51:58.504590 18301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:58.505731 18301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:58.507483 18301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:58.520020 18312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:58.520045 18314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:58.521872 18311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:58.522485 18301 server_base.cc:1034] running on GCE node
I20260810 00:51:58.523032 18301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:58.523901 18301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:58.525094 18301 hybrid_clock.cc:648] HybridClock initialized: now 1786323118525042 us; error 45 us; skew 500 ppm
I20260810 00:51:58.525389 18301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:51:58.527463 18301 webserver.cc:460] Webserver started at http://127.10.97.66:38843/ using document root <none> and password file <none>
I20260810 00:51:58.528124 18301 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:58.528235 18301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:58.528543 18301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:58.531114 18301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "b13226a7b29348888e9fe98d5deebd1d"
format_stamp: "Formatted at 2026-08-10 00:51:58 on dist-test-slave-jlzn"
server_key: "8c0eed1f09263379c5b28ec0a26724af"
server_key_iv: "952ef489eafe391e7a8377af88f6a52b"
server_key_version: "encryptionkey@0"
I20260810 00:51:58.531905 18301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "b13226a7b29348888e9fe98d5deebd1d"
format_stamp: "Formatted at 2026-08-10 00:51:58 on dist-test-slave-jlzn"
server_key: "8c0eed1f09263379c5b28ec0a26724af"
server_key_iv: "952ef489eafe391e7a8377af88f6a52b"
server_key_version: "encryptionkey@0"
I20260810 00:51:58.536772 18301 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 00:51:58.539947 18325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:58.541071 18301 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:51:58.541226 18301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "b13226a7b29348888e9fe98d5deebd1d"
format_stamp: "Formatted at 2026-08-10 00:51:58 on dist-test-slave-jlzn"
server_key: "8c0eed1f09263379c5b28ec0a26724af"
server_key_iv: "952ef489eafe391e7a8377af88f6a52b"
server_key_version: "encryptionkey@0"
I20260810 00:51:58.541414 18301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:58.546461 18301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:58.547145 18301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:58.547391 18301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:58.548286 18301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:58.549522 18301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:58.549607 18301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:58.549682 18301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:58.549739 18301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:58.578259 18301 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:42483
I20260810 00:51:58.578320 18509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:42483 every 8 connection(s)
I20260810 00:51:58.579427 18301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:51:58.586900 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:58.581707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:55895 (local address 127.10.97.126:33729)
0810 00:51:58.581875 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:58.581880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:58.583178 (+ 1298us) server_negotiation.cc:407] Connection header received
0810 00:51:58.584844 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:58.584850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:58.584912 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:58.585001 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:58.586180 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:58.586186 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:58.586198 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:51:58.586215 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:58.586248 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:51:58.586259 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:58.586364 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:58.586544 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 00:51:58.586615 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 00:51:58.587726 18515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:58.582158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33729 (local address 127.10.97.66:55895)
0810 00:51:58.583015 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:58.583058 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:58.584577 (+ 1519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:58.585233 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:58.585240 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:58.585324 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:58.585903 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:51:58.585905 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:58.585939 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:51:58.586019 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:58.586348 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:58.586359 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:58.586367 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 00:51:58.586615 (+ 248us) client_negotiation.cc:240] Negotiation successful
0810 00:51:58.586853 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":265,"threads_started":1}
I20260810 00:51:58.588135 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 18301
I20260810 00:51:58.588328 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:51:58.588909 10629 external_mini_cluster.cc:1428] Setting key a624c735230c1953ef98a4ea884d0e85
I20260810 00:51:58.589167 18510 heartbeater.cc:347] Connected to a master server at 127.10.97.126:33729
I20260810 00:51:58.589440 18510 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:58.590039 18510 heartbeater.cc:511] Master 127.10.97.126:33729 requested a full tablet report, sending...
I20260810 00:51:58.591393 18013 ts_manager.cc:194] Registered new tserver with Master: b13226a7b29348888e9fe98d5deebd1d (127.10.97.66:42483)
I20260810 00:51:58.592031 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:33729
--builtin_ntp_servers=127.10.97.84:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 00:51:58.904006 18516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:51:58.904247 18516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:51:58.904364 18516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:51:58.912143 18516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:51:58.912228 18516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:51:58.912369 18516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:51:58.921003 18516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:51:58.922102 18516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:51:58.923714 18516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:51:58.935925 18525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:58.935971 18529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:51:58.936005 18524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:51:58.936672 18516 server_base.cc:1034] running on GCE node
I20260810 00:51:58.937156 18516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:51:58.937959 18516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:51:58.939157 18516 hybrid_clock.cc:648] HybridClock initialized: now 1786323118939106 us; error 41 us; skew 500 ppm
I20260810 00:51:58.939435 18516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:51:58.941439 18516 webserver.cc:460] Webserver started at http://127.10.97.67:33479/ using document root <none> and password file <none>
I20260810 00:51:58.942041 18516 fs_manager.cc:359] Metadata directory not provided
I20260810 00:51:58.942135 18516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:51:58.942440 18516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:51:58.945076 18516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "9f9090fcaa3b4dbcb28cfc96e4248875"
format_stamp: "Formatted at 2026-08-10 00:51:58 on dist-test-slave-jlzn"
server_key: "a5dbf21dabef061ff002454c7e1095f5"
server_key_iv: "e77fc41d2b6c7a63c6f592eb0c07cb66"
server_key_version: "encryptionkey@0"
I20260810 00:51:58.945828 18516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "9f9090fcaa3b4dbcb28cfc96e4248875"
format_stamp: "Formatted at 2026-08-10 00:51:58 on dist-test-slave-jlzn"
server_key: "a5dbf21dabef061ff002454c7e1095f5"
server_key_iv: "e77fc41d2b6c7a63c6f592eb0c07cb66"
server_key_version: "encryptionkey@0"
I20260810 00:51:58.950373 18516 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:51:58.953455 18538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:58.954576 18516 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:51:58.954721 18516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "9f9090fcaa3b4dbcb28cfc96e4248875"
format_stamp: "Formatted at 2026-08-10 00:51:58 on dist-test-slave-jlzn"
server_key: "a5dbf21dabef061ff002454c7e1095f5"
server_key_iv: "e77fc41d2b6c7a63c6f592eb0c07cb66"
server_key_version: "encryptionkey@0"
I20260810 00:51:58.954906 18516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:51:58.960093 18516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:51:58.960773 18516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:51:58.961006 18516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:51:58.961812 18516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:51:58.963054 18516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:51:58.963147 18516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:58.963250 18516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:51:58.963325 18516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:51:58.992167 18516 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:37037
I20260810 00:51:58.992224 18726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:37037 every 8 connection(s)
I20260810 00:51:58.993350 18516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:51:59.000357 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:58.995549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:44893 (local address 127.10.97.126:33729)
0810 00:51:58.995742 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:58.995747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:58.997023 (+ 1276us) server_negotiation.cc:407] Connection header received
0810 00:51:58.998540 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:58.998544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:58.998594 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:58.998684 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:58.999784 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:58.999787 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:58.999789 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:51:58.999826 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:58.999862 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:51:58.999873 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:58.999969 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:59.000101 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 00:51:59.000151 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 00:51:59.001149 18732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:58.996051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33729 (local address 127.10.97.67:44893)
0810 00:51:58.996889 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:58.996922 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:58.998329 (+ 1407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:58.998848 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:58.998855 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:58.998935 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:58.999541 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:51:58.999543 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:58.999577 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:51:58.999665 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:58.999948 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:58.999955 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:58.999959 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 00:51:59.000161 (+ 202us) client_negotiation.cc:240] Negotiation successful
0810 00:51:59.000330 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":244,"threads_started":1}
I20260810 00:51:59.001492 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 18516
I20260810 00:51:59.001667 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:51:59.002147 10629 external_mini_cluster.cc:1428] Setting key 8ff1d83781c52c35da286f66543abfdf
I20260810 00:51:59.002415 18727 heartbeater.cc:347] Connected to a master server at 127.10.97.126:33729
I20260810 00:51:59.002709 18727 heartbeater.cc:464] Registering TS with master...
I20260810 00:51:59.003336 18727 heartbeater.cc:511] Master 127.10.97.126:33729 requested a full tablet report, sending...
I20260810 00:51:59.004878 18012 ts_manager.cc:194] Registered new tserver with Master: 9f9090fcaa3b4dbcb28cfc96e4248875 (127.10.97.67:37037)
I20260810 00:51:59.016773 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:51:59.023208 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.020932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51958 (local address 127.10.97.126:33729)
0810 00:51:59.021068 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:59.021072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:59.021573 (+ 501us) server_negotiation.cc:407] Connection header received
0810 00:51:59.021727 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:59.021732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:59.021783 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:59.021857 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:59.022636 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:59.022642 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:59.022647 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:51:59.022664 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:59.022693 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:51:59.022703 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:59.022803 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:59.022902 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 00:51:59.022957 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 00:51:59.026767 18012 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51958:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:51:59.035173 18012 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:51:59.053649 18755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.048147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:42483 (local address 127.0.0.1:56898)
0810 00:51:59.048748 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:59.048765 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:59.048858 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:59.050995 (+ 2137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:59.051003 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:59.051090 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:59.052268 (+ 1178us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:51:59.052272 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:59.052302 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:51:59.052390 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:59.052941 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:59.052953 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:59.052961 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 00:51:59.053127 (+ 166us) client_negotiation.cc:240] Negotiation successful
0810 00:51:59.053216 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":397,"threads_started":1}
I20260810 00:51:59.054589 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.048843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56898 (local address 127.10.97.66:42483)
0810 00:51:59.049417 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:59.049426 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:59.049456 (+ 30us) server_negotiation.cc:407] Connection header received
0810 00:51:59.049528 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:59.049536 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:59.049766 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:59.049910 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:59.052581 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:59.052591 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:59.052607 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:51:59.052648 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:59.052710 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:51:59.052730 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:59.052885 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:59.054092 (+ 1207us) server_negotiation.cc:299] Negotiation successful
0810 00:51:59.054195 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":342,"threads_started":1}
I20260810 00:51:59.058600 18398 tablet_service.cc:1467] Processing CreateTablet for tablet 1f63dd36ebe54d3aa092beffffaa9967 (DEFAULT_TABLE table=test-table [id=f3d07df3dd394c7e96817d1e20a8b69f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:59.060122 18398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f63dd36ebe54d3aa092beffffaa9967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:59.062008 18754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.047463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41226 (local address 127.10.97.67:37037)
0810 00:51:59.059466 (+ 12003us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:59.059473 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:59.059505 (+ 32us) server_negotiation.cc:407] Connection header received
0810 00:51:59.059567 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:59.059574 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:59.059765 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:59.060146 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:59.061211 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:59.061219 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:59.061231 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:51:59.061248 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:59.061301 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:51:59.061320 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:59.061498 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:59.061561 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 00:51:59.061650 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11846,"thread_start_us":12422,"threads_started":1}
I20260810 00:51:59.062027 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.047332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:37037 (local address 127.0.0.1:41226)
0810 00:51:59.048163 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:59.048202 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:59.048347 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:59.060154 (+ 11807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:59.060160 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:59.060186 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:59.060891 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:51:59.060895 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:59.060913 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:51:59.060963 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:59.061427 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:59.061432 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:59.061437 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 00:51:59.061572 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 00:51:59.061625 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":373,"threads_started":1}
I20260810 00:51:59.064543 18755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.048437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:40977 (local address 127.0.0.1:54334)
0810 00:51:59.053845 (+ 5408us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:59.053861 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:59.053926 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:59.063051 (+ 9125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:59.063056 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:59.063089 (+ 33us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:59.063550 (+ 461us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:51:59.063554 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:59.063569 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:51:59.063611 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:59.064120 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:59.064136 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:59.064141 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 00:51:59.064236 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 00:51:59.064281 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5328,"thread_start_us":10851,"threads_started":1}
I20260810 00:51:59.065214 18761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.049390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54334 (local address 127.10.97.65:40977)
0810 00:51:59.062591 (+ 13201us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:59.062598 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:59.062627 (+ 29us) server_negotiation.cc:407] Connection header received
0810 00:51:59.062692 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:59.062700 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:59.062884 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:59.063028 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:59.063832 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:59.063840 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:59.063851 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:51:59.063874 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:59.063925 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:51:59.063942 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:59.064063 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:59.064804 (+ 741us) server_negotiation.cc:299] Negotiation successful
0810 00:51:59.064891 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13052,"thread_start_us":12922,"threads_started":1}
I20260810 00:51:59.065804 18627 tablet_service.cc:1467] Processing CreateTablet for tablet 1f63dd36ebe54d3aa092beffffaa9967 (DEFAULT_TABLE table=test-table [id=f3d07df3dd394c7e96817d1e20a8b69f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:59.067063 18627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f63dd36ebe54d3aa092beffffaa9967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:59.068594 18199 tablet_service.cc:1467] Processing CreateTablet for tablet 1f63dd36ebe54d3aa092beffffaa9967 (DEFAULT_TABLE table=test-table [id=f3d07df3dd394c7e96817d1e20a8b69f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:51:59.069859 18199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f63dd36ebe54d3aa092beffffaa9967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:51:59.072374 18764 tablet_bootstrap.cc:492] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d: Bootstrap starting.
I20260810 00:51:59.075285 18764 tablet_bootstrap.cc:654] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:59.076270 18764 log.cc:824] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:59.080735 18768 tablet_bootstrap.cc:492] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875: Bootstrap starting.
I20260810 00:51:59.082355 18764 tablet_bootstrap.cc:492] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d: No bootstrap required, opened a new log
I20260810 00:51:59.082595 18764 ts_tablet_manager.cc:1397] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260810 00:51:59.082872 18769 tablet_bootstrap.cc:492] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4: Bootstrap starting.
I20260810 00:51:59.084056 18768 tablet_bootstrap.cc:654] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:59.085208 18768 log.cc:824] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:59.085455 18769 tablet_bootstrap.cc:654] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4: Neither blocks nor log segments found. Creating new log.
I20260810 00:51:59.086529 18769 log.cc:824] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4: Log is configured to *not* fsync() on all Append() calls
I20260810 00:51:59.089161 18768 tablet_bootstrap.cc:492] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875: No bootstrap required, opened a new log
I20260810 00:51:59.089254 18769 tablet_bootstrap.cc:492] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4: No bootstrap required, opened a new log
I20260810 00:51:59.089404 18768 ts_tablet_manager.cc:1397] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260810 00:51:59.089491 18769 ts_tablet_manager.cc:1397] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260810 00:51:59.092664 18764 raft_consensus.cc:348] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9090fcaa3b4dbcb28cfc96e4248875" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37037 } } peers { permanent_uuid: "b552fbcf05734ef1a00e65e3dc690bd4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40977 } } peers { permanent_uuid: "b13226a7b29348888e9fe98d5deebd1d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42483 } }
I20260810 00:51:59.093096 18764 raft_consensus.cc:374] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:59.093194 18764 raft_consensus.cc:729] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b13226a7b29348888e9fe98d5deebd1d, State: Initialized, Role: FOLLOWER
I20260810 00:51:59.093781 18764 consensus_queue.cc:260] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9090fcaa3b4dbcb28cfc96e4248875" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37037 } } peers { permanent_uuid: "b552fbcf05734ef1a00e65e3dc690bd4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40977 } } peers { permanent_uuid: "b13226a7b29348888e9fe98d5deebd1d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42483 } }
I20260810 00:51:59.096349 18768 raft_consensus.cc:348] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9090fcaa3b4dbcb28cfc96e4248875" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37037 } } peers { permanent_uuid: "b552fbcf05734ef1a00e65e3dc690bd4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40977 } } peers { permanent_uuid: "b13226a7b29348888e9fe98d5deebd1d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42483 } }
I20260810 00:51:59.096776 18768 raft_consensus.cc:374] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:59.096884 18768 raft_consensus.cc:729] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f9090fcaa3b4dbcb28cfc96e4248875, State: Initialized, Role: FOLLOWER
I20260810 00:51:59.097494 18510 heartbeater.cc:503] Master 127.10.97.126:33729 was elected leader, sending a full tablet report...
I20260810 00:51:59.097420 18768 consensus_queue.cc:260] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9090fcaa3b4dbcb28cfc96e4248875" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37037 } } peers { permanent_uuid: "b552fbcf05734ef1a00e65e3dc690bd4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40977 } } peers { permanent_uuid: "b13226a7b29348888e9fe98d5deebd1d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42483 } }
I20260810 00:51:59.098409 18727 heartbeater.cc:503] Master 127.10.97.126:33729 was elected leader, sending a full tablet report...
I20260810 00:51:59.099045 18768 ts_tablet_manager.cc:1428] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260810 00:51:59.099200 18764 ts_tablet_manager.cc:1428] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d: Time spent starting tablet: real 0.016s user 0.011s sys 0.005s
I20260810 00:51:59.100108 18769 raft_consensus.cc:348] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9090fcaa3b4dbcb28cfc96e4248875" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37037 } } peers { permanent_uuid: "b552fbcf05734ef1a00e65e3dc690bd4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40977 } } peers { permanent_uuid: "b13226a7b29348888e9fe98d5deebd1d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42483 } }
I20260810 00:51:59.100504 18769 raft_consensus.cc:374] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:51:59.100607 18769 raft_consensus.cc:729] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b552fbcf05734ef1a00e65e3dc690bd4, State: Initialized, Role: FOLLOWER
I20260810 00:51:59.101169 18769 consensus_queue.cc:260] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9090fcaa3b4dbcb28cfc96e4248875" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37037 } } peers { permanent_uuid: "b552fbcf05734ef1a00e65e3dc690bd4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40977 } } peers { permanent_uuid: "b13226a7b29348888e9fe98d5deebd1d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42483 } }
I20260810 00:51:59.102144 18297 heartbeater.cc:503] Master 127.10.97.126:33729 was elected leader, sending a full tablet report...
I20260810 00:51:59.102788 18769 ts_tablet_manager.cc:1428] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260810 00:51:59.169497 18298 tablet.cc:2369] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:51:59.244918 18730 tablet.cc:2369] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:51:59.316907 18776 raft_consensus.cc:482] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:51:59.317220 18776 raft_consensus.cc:504] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9090fcaa3b4dbcb28cfc96e4248875" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37037 } } peers { permanent_uuid: "b552fbcf05734ef1a00e65e3dc690bd4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40977 } } peers { permanent_uuid: "b13226a7b29348888e9fe98d5deebd1d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42483 } }
I20260810 00:51:59.318935 18776 leader_election.cc:288] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f9090fcaa3b4dbcb28cfc96e4248875 (127.10.97.67:37037), b13226a7b29348888e9fe98d5deebd1d (127.10.97.66:42483)
I20260810 00:51:59.322255 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.319458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:38829 (local address 127.10.97.66:42483)
0810 00:51:59.319660 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:59.319666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:59.320284 (+ 618us) server_negotiation.cc:407] Connection header received
0810 00:51:59.320415 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:59.320420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:59.320482 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:59.320553 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:59.321630 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:59.321636 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:59.321640 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:51:59.321660 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:59.321699 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:51:59.321715 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:59.321806 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:59.321930 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 00:51:59.321995 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 00:51:59.322278 18754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.319340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:48537 (local address 127.10.97.67:37037)
0810 00:51:59.319521 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:59.319527 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:59.320340 (+ 813us) server_negotiation.cc:407] Connection header received
0810 00:51:59.320465 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:59.320469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:59.320517 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:59.320608 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:59.321707 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:59.321711 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:59.321714 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:51:59.321728 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:59.321762 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:51:59.321776 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:59.321894 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:59.321988 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 00:51:59.322041 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 00:51:59.322461 18777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.319222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:42483 (local address 127.10.97.65:38829)
0810 00:51:59.320169 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:59.320186 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:59.320295 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:59.320606 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:59.320611 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:59.320634 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:59.321437 (+ 803us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:51:59.321439 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:59.321453 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:51:59.321493 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:59.321813 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:59.321818 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:59.321823 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 00:51:59.321959 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 00:51:59.322016 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":493,"threads_started":1}
I20260810 00:51:59.322825 18779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.319308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:37037 (local address 127.10.97.65:48537)
0810 00:51:59.320227 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:59.320243 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:51:59.320357 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:51:59.320656 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:51:59.320660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:51:59.320681 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:51:59.321514 (+ 833us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:51:59.321517 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:51:59.321530 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:51:59.321573 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:51:59.321870 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:51:59.321874 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:51:59.321879 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 00:51:59.322029 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 00:51:59.322080 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"thread_start_us":621,"threads_started":1}
I20260810 00:51:59.323426 18430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f63dd36ebe54d3aa092beffffaa9967" candidate_uuid: "b552fbcf05734ef1a00e65e3dc690bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13226a7b29348888e9fe98d5deebd1d" is_pre_election: true
I20260810 00:51:59.323742 18662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f63dd36ebe54d3aa092beffffaa9967" candidate_uuid: "b552fbcf05734ef1a00e65e3dc690bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f9090fcaa3b4dbcb28cfc96e4248875" is_pre_election: true
I20260810 00:51:59.323897 18430 raft_consensus.cc:2454] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b552fbcf05734ef1a00e65e3dc690bd4 in term 0.
I20260810 00:51:59.324225 18662 raft_consensus.cc:2454] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b552fbcf05734ef1a00e65e3dc690bd4 in term 0.
I20260810 00:51:59.324553 18129 leader_election.cc:302] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b13226a7b29348888e9fe98d5deebd1d, b552fbcf05734ef1a00e65e3dc690bd4; no voters:
I20260810 00:51:59.324970 18776 raft_consensus.cc:2781] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:51:59.325122 18776 raft_consensus.cc:482] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:51:59.325230 18776 raft_consensus.cc:3037] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:59.327289 18776 raft_consensus.cc:504] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9090fcaa3b4dbcb28cfc96e4248875" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37037 } } peers { permanent_uuid: "b552fbcf05734ef1a00e65e3dc690bd4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40977 } } peers { permanent_uuid: "b13226a7b29348888e9fe98d5deebd1d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42483 } }
I20260810 00:51:59.327790 18776 leader_election.cc:288] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 [CANDIDATE]: Term 1 election: Requested vote from peers 9f9090fcaa3b4dbcb28cfc96e4248875 (127.10.97.67:37037), b13226a7b29348888e9fe98d5deebd1d (127.10.97.66:42483)
I20260810 00:51:59.328182 18662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f63dd36ebe54d3aa092beffffaa9967" candidate_uuid: "b552fbcf05734ef1a00e65e3dc690bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f9090fcaa3b4dbcb28cfc96e4248875"
I20260810 00:51:59.328277 18430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f63dd36ebe54d3aa092beffffaa9967" candidate_uuid: "b552fbcf05734ef1a00e65e3dc690bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13226a7b29348888e9fe98d5deebd1d"
I20260810 00:51:59.328357 18662 raft_consensus.cc:3037] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:59.328473 18430 raft_consensus.cc:3037] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:51:59.330461 18662 raft_consensus.cc:2454] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b552fbcf05734ef1a00e65e3dc690bd4 in term 1.
I20260810 00:51:59.330648 18430 raft_consensus.cc:2454] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b552fbcf05734ef1a00e65e3dc690bd4 in term 1.
I20260810 00:51:59.330837 18130 leader_election.cc:302] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f9090fcaa3b4dbcb28cfc96e4248875, b552fbcf05734ef1a00e65e3dc690bd4; no voters:
I20260810 00:51:59.331122 18776 raft_consensus.cc:2781] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 [term 1 FOLLOWER]: Leader election won for term 1
W20260810 00:51:59.331544 18512 tablet.cc:2369] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:51:59.331578 18776 raft_consensus.cc:686] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 [term 1 LEADER]: Becoming Leader. State: Replica: b552fbcf05734ef1a00e65e3dc690bd4, State: Running, Role: LEADER
I20260810 00:51:59.332109 18776 consensus_queue.cc:237] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9090fcaa3b4dbcb28cfc96e4248875" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37037 } } peers { permanent_uuid: "b552fbcf05734ef1a00e65e3dc690bd4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40977 } } peers { permanent_uuid: "b13226a7b29348888e9fe98d5deebd1d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42483 } }
I20260810 00:51:59.336490 18013 catalog_manager.cc:5495] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b552fbcf05734ef1a00e65e3dc690bd4 (127.10.97.65). New cstate: current_term: 1 leader_uuid: "b552fbcf05734ef1a00e65e3dc690bd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9090fcaa3b4dbcb28cfc96e4248875" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b552fbcf05734ef1a00e65e3dc690bd4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b13226a7b29348888e9fe98d5deebd1d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42483 } health_report { overall_health: UNKNOWN } } }
I20260810 00:51:59.414784 18761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.412718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54338 (local address 127.10.97.65:40977)
0810 00:51:59.412891 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:59.412898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:59.412917 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:51:59.413100 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:59.413104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:59.413217 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:59.413318 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:59.414255 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:59.414261 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:59.414266 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:51:59.414284 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:59.414316 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:51:59.414328 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:59.414416 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:59.414473 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 00:51:59.414517 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 00:51:59.422374 18430 raft_consensus.cc:1261] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d [term 1 FOLLOWER]: Refusing update from remote peer b552fbcf05734ef1a00e65e3dc690bd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:51:59.422513 18662 raft_consensus.cc:1261] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875 [term 1 FOLLOWER]: Refusing update from remote peer b552fbcf05734ef1a00e65e3dc690bd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:51:59.423336 18776 consensus_queue.cc:1035] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f9090fcaa3b4dbcb28cfc96e4248875" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:51:59.423609 18780 consensus_queue.cc:1035] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13226a7b29348888e9fe98d5deebd1d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:51:59.434754 18784 mvcc.cc:205] Tried to move back new op lower bound from 7316779497145036800 to 7316779496787898368. Current Snapshot: MvccSnapshot[applied={T|T < 7316779497145036800}]
I20260810 00:51:59.437959 18787 mvcc.cc:205] Tried to move back new op lower bound from 7316779497145036800 to 7316779496787898368. Current Snapshot: MvccSnapshot[applied={T|T < 7316779497145036800}]
I20260810 00:51:59.445899 18011 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:51:59.446192 18011 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:51:59.449883 18011 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f02f2b5a46034a7dbf90dbd6a67ebe51: Sending DeleteTablet for 3 replicas of tablet 1f63dd36ebe54d3aa092beffffaa9967
I20260810 00:51:59.450834 18627 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f63dd36ebe54d3aa092beffffaa9967 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:59 UTC) from {username='slave'} at 127.0.0.1:41226
I20260810 00:51:59.450966 18199 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f63dd36ebe54d3aa092beffffaa9967 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:59 UTC) from {username='slave'} at 127.0.0.1:54334
I20260810 00:51:59.451478 18398 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f63dd36ebe54d3aa092beffffaa9967 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:51:59 UTC) from {username='slave'} at 127.0.0.1:56898
I20260810 00:51:59.453472 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 18107
I20260810 00:51:59.469905 18798 tablet_replica.cc:330] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875: stopping tablet replica
I20260810 00:51:59.470352 18798 raft_consensus.cc:2229] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:51:59.470853 18798 raft_consensus.cc:2258] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:59.474484 18798 ts_tablet_manager.cc:1905] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:59.474735 18800 tablet_replica.cc:330] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d: stopping tablet replica
I20260810 00:51:59.475142 18800 raft_consensus.cc:2229] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:51:59.475607 18800 raft_consensus.cc:2258] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:51:59.476662 18761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.454199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54348 (local address 127.10.97.65:40977)
0810 00:51:59.454368 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:59.454374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:59.473972 (+ 19598us) server_negotiation.cc:407] Connection header received
0810 00:51:59.474210 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:59.474216 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:59.474285 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:59.474390 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:59.475849 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:59.475856 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:59.475860 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:51:59.475881 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:59.475925 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:51:59.475941 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:59.476039 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:59.476328 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 00:51:59.476394 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 00:51:59.477735 18800 ts_tablet_manager.cc:1905] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:51:59.477840 18291 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:59.482353 18798 ts_tablet_manager.cc:1918] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:51:59.482509 18798 log.cc:1196] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/1f63dd36ebe54d3aa092beffffaa9967
I20260810 00:51:59.483076 18798 ts_tablet_manager.cc:1939] T 1f63dd36ebe54d3aa092beffffaa9967 P 9f9090fcaa3b4dbcb28cfc96e4248875: Deleting consensus metadata
I20260810 00:51:59.485033 17994 catalog_manager.cc:4841] TS 9f9090fcaa3b4dbcb28cfc96e4248875 (127.10.97.67:37037): tablet 1f63dd36ebe54d3aa092beffffaa9967 (table test-table [id=f3d07df3dd394c7e96817d1e20a8b69f]) successfully deleted
I20260810 00:51:59.485939 18800 ts_tablet_manager.cc:1918] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:51:59.486063 18800 log.cc:1196] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/1f63dd36ebe54d3aa092beffffaa9967
I20260810 00:51:59.486472 18800 ts_tablet_manager.cc:1939] T 1f63dd36ebe54d3aa092beffffaa9967 P b13226a7b29348888e9fe98d5deebd1d: Deleting consensus metadata
I20260810 00:51:59.487983 17993 catalog_manager.cc:4841] TS b13226a7b29348888e9fe98d5deebd1d (127.10.97.66:42483): tablet 1f63dd36ebe54d3aa092beffffaa9967 (table test-table [id=f3d07df3dd394c7e96817d1e20a8b69f]) successfully deleted
I20260810 00:51:59.571673 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 18107
I20260810 00:51:59.571699 18799 tablet_replica.cc:330] T 1f63dd36ebe54d3aa092beffffaa9967 P b552fbcf05734ef1a00e65e3dc690bd4: stopping tablet replica
I20260810 00:51:59.594496 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 18301
W20260810 00:51:59.594590 17994 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.97.65:40977 (error 108)
I20260810 00:51:59.596683 18804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.595915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:40977 (local address 127.0.0.1:54362)
0810 00:51:59.596303 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:59.596505 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:40977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":206,"threads_started":1}
W20260810 00:51:59.596952 17994 catalog_manager.cc:4584] TS b552fbcf05734ef1a00e65e3dc690bd4 (127.10.97.65:40977): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1f63dd36ebe54d3aa092beffffaa9967: Network error: Client connection negotiation failed: client connection to 127.10.97.65:40977: connect: Connection refused (error 111)
I20260810 00:51:59.597498 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.595210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56908 (local address 127.10.97.66:42483)
0810 00:51:59.595396 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:59.595403 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:59.595545 (+ 142us) server_negotiation.cc:407] Connection header received
0810 00:51:59.595749 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:59.595755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:59.595861 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:59.595945 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:59.596851 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:59.596857 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:59.596862 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:51:59.596881 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:59.596920 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:51:59.596936 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:59.597080 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:59.597179 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 00:51:59.597240 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 00:51:59.598068 18502 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:59.650570 18804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.650095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:40977 (local address 127.0.0.1:54376)
0810 00:51:59.650297 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:59.650407 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:40977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 00:51:59.651610 18804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.651203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:40977 (local address 127.0.0.1:54392)
0810 00:51:59.651337 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:59.651440 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:40977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 00:51:59.683252 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 18301
I20260810 00:51:59.705844 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 18516
I20260810 00:51:59.708250 18754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.706492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41238 (local address 127.10.97.67:37037)
0810 00:51:59.706643 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 00:51:59.706647 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:51:59.706666 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:51:59.706801 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:51:59.706805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:51:59.706884 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:51:59.706965 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:51:59.707744 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:51:59.707748 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:51:59.707751 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:51:59.707761 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:51:59.707785 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:51:59.707795 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:51:59.707934 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 00:51:59.707973 (+ 39us) server_negotiation.cc:299] Negotiation successful
0810 00:51:59.708017 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 00:51:59.708847 18721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:59.712670 18804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.712247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:40977 (local address 127.0.0.1:54404)
0810 00:51:59.712426 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:59.712517 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:40977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 00:51:59.713675 18804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.713275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:40977 (local address 127.0.0.1:54420)
0810 00:51:59.713435 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:59.713518 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:40977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 00:51:59.793133 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 18516
I20260810 00:51:59.796703 18804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.793654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:40977 (local address 127.0.0.1:54432)
0810 00:51:59.796479 (+ 2825us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:59.796572 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:40977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2698}
I20260810 00:51:59.797915 18804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:51:59.797368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:40977 (local address 127.0.0.1:54440)
0810 00:51:59.797616 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 00:51:59.797740 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:40977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260810 00:51:59.812763 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 17973
I20260810 00:51:59.813709 18066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:51:59.915707 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 17973
2026-08-10T00:51:59Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2714 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 00:51:59.959407 10629 mini_oidc.cc:152] Starting JWKS server
I20260810 00:51:59.962319 10629 webserver.cc:460] Webserver started at https://127.0.0.1:41521/ using document root <none> and password file <none>
I20260810 00:51:59.962592 10629 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 00:51:59.963187 10629 webserver.cc:460] Webserver started at http://127.0.0.1:33497/ using document root <none> and password file <none>
2026-08-10T00:51:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:51:59Z Disabled control of system clock
I20260810 00:51:59.991524 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:37145
--webserver_interface=127.10.97.126
--webserver_port=0
--builtin_ntp_servers=127.10.97.84:45411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:37145
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41521/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0/testchainca.cert with env {}
W20260810 00:52:00.298542 18820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:00.298792 18820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:00.298880 18820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:00.305683 18820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:52:00.305775 18820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:00.305819 18820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:00.305873 18820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:52:00.305914 18820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 00:52:00.305971 18820 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 00:52:00.306030 18820 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41521/jwks.json
I20260810 00:52:00.315152 18820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:45411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:37145
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:37145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41521/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:00.316219 18820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:00.317346 18820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:00.328300 18825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:00.328341 18830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:00.328382 18828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:00.329295 18820 server_base.cc:1034] running on GCE node
I20260810 00:52:00.329993 18820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:00.331188 18820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:00.332423 18820 hybrid_clock.cc:648] HybridClock initialized: now 1786323120332379 us; error 26 us; skew 500 ppm
I20260810 00:52:00.332762 18820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:52:00.334851 18820 webserver.cc:460] Webserver started at http://127.10.97.126:38945/ using document root <none> and password file <none>
I20260810 00:52:00.335453 18820 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:00.335568 18820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:00.335902 18820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:00.338531 18820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "7918b9c54dc640289c48a430a1afd5f6"
format_stamp: "Formatted at 2026-08-10 00:52:00 on dist-test-slave-jlzn"
server_key: "71fda2893d83eaf0cbc7065a0a567382"
server_key_iv: "731c96a42e25df7c9505ac1dc6eb98c5"
server_key_version: "encryptionkey@0"
I20260810 00:52:00.339268 18820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "7918b9c54dc640289c48a430a1afd5f6"
format_stamp: "Formatted at 2026-08-10 00:52:00 on dist-test-slave-jlzn"
server_key: "71fda2893d83eaf0cbc7065a0a567382"
server_key_iv: "731c96a42e25df7c9505ac1dc6eb98c5"
server_key_version: "encryptionkey@0"
I20260810 00:52:00.344063 18820 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260810 00:52:00.347231 18839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:00.348430 18820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:52:00.348616 18820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "7918b9c54dc640289c48a430a1afd5f6"
format_stamp: "Formatted at 2026-08-10 00:52:00 on dist-test-slave-jlzn"
server_key: "71fda2893d83eaf0cbc7065a0a567382"
server_key_iv: "731c96a42e25df7c9505ac1dc6eb98c5"
server_key_version: "encryptionkey@0"
I20260810 00:52:00.348811 18820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:00.376334 18820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:00.377050 18820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:00.377287 18820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:00.393224 18820 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:37145
I20260810 00:52:00.393271 18921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:37145 every 8 connection(s)
I20260810 00:52:00.394681 18820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:52:00.396631 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 18820
I20260810 00:52:00.396783 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:52:00.397408 10629 external_mini_cluster.cc:1428] Setting key 5bd788a317a9c0dae1ed2c70207c59a8
I20260810 00:52:00.398227 18923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:00.411042 18928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:00.399082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38476 (local address 127.10.97.126:37145)
0810 00:52:00.400113 (+ 1031us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:00.400127 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:00.400177 (+ 50us) server_negotiation.cc:407] Connection header received
0810 00:52:00.401705 (+ 1528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:00.401731 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:00.402278 (+ 547us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:00.402684 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:00.403160 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:00.405117 (+ 1957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:00.406397 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:00.407338 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:00.407512 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:00.408511 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:00.408521 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:00.408541 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:52:00.409218 (+ 677us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:00.409311 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:52:00.409343 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:00.409454 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:00.409643 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 00:52:00.409942 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":312,"threads_started":1}
I20260810 00:52:00.412281 18923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6: Bootstrap starting.
I20260810 00:52:00.414696 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:52:00.415473 18923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:00.416412 18923 log.cc:824] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:00.418849 18923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6: No bootstrap required, opened a new log
I20260810 00:52:00.425570 18923 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7918b9c54dc640289c48a430a1afd5f6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 37145 } }
I20260810 00:52:00.425855 18923 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:00.425944 18923 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7918b9c54dc640289c48a430a1afd5f6, State: Initialized, Role: FOLLOWER
I20260810 00:52:00.426479 18923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7918b9c54dc640289c48a430a1afd5f6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 37145 } }
I20260810 00:52:00.426757 18923 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:52:00.426885 18923 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:52:00.427038 18923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:00.428979 18923 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7918b9c54dc640289c48a430a1afd5f6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 37145 } }
I20260810 00:52:00.429415 18923 leader_election.cc:302] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7918b9c54dc640289c48a430a1afd5f6; no voters:
I20260810 00:52:00.430823 18923 leader_election.cc:288] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:52:00.430914 18932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:52:00.431556 18932 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6 [term 1 LEADER]: Becoming Leader. State: Replica: 7918b9c54dc640289c48a430a1afd5f6, State: Running, Role: LEADER
I20260810 00:52:00.432061 18932 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7918b9c54dc640289c48a430a1afd5f6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 37145 } }
I20260810 00:52:00.432660 18923 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:52:00.434437 18936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7918b9c54dc640289c48a430a1afd5f6. Latest consensus state: current_term: 1 leader_uuid: "7918b9c54dc640289c48a430a1afd5f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7918b9c54dc640289c48a430a1afd5f6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 37145 } } }
I20260810 00:52:00.434362 18934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7918b9c54dc640289c48a430a1afd5f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7918b9c54dc640289c48a430a1afd5f6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 37145 } } }
I20260810 00:52:00.434803 18934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:00.434803 18936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:00.435788 18944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:52:00.440845 18944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:52:00.450011 18944 catalog_manager.cc:1347] Generated new cluster ID: 31f2c1a0bf334d15b52c625a2788e393
I20260810 00:52:00.450124 18944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:52:00.460361 18944 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:52:00.461532 18944 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:52:00.470492 18944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7918b9c54dc640289c48a430a1afd5f6: Generated new TSK 0
I20260810 00:52:00.471140 18944 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 00:52:00.513588 18970 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 00:52:00.513797 18970 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 00:52:00.514616 18928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:00.494185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38480 (local address 127.10.97.126:37145)
0810 00:52:00.494408 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:00.494415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:00.494816 (+ 401us) server_negotiation.cc:407] Connection header received
0810 00:52:00.495154 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:00.495157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:00.495228 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:00.495344 (+ 116us) server_negotiation.cc:226] Negotiated authn=JWT
0810 00:52:00.495955 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:00.496660 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:00.498680 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:00.499101 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:00.499203 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:00.499772 (+ 569us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 00:52:00.514137 (+ 14365us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41521/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 00:52:00.514384 (+ 247us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38480: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41521/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":66}
W20260810 00:52:00.514726 18928 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38480: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41521/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 00:52:00.516513 18969 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:00.494026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:37145 (local address 127.0.0.1:38480)
0810 00:52:00.494712 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:00.494736 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:00.494908 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:00.495391 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:00.495397 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:00.495410 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0810 00:52:00.495797 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:00.495834 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:00.496787 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:00.496793 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:00.498551 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:00.498566 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:00.499264 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:00.499270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:00.499449 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:00.499523 (+ 74us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 00:52:00.514371 (+ 14848us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41521/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 00:52:00.514683 (+ 312us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.97.126:37145: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41521/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":351,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.10.97.126:37145: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41521/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 00:52:00.518162 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 18820
I20260810 00:52:00.519112 18916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:00.609494 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 18820
2026-08-10T00:52:00Z chronyd exiting
I20260810 00:52:00.666620 10629 test_util.cc:175] -----------------------------------------------
I20260810 00:52:00.666742 10629 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786323077328248-10629-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (711 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 00:52:00.670384 10629 mini_oidc.cc:152] Starting JWKS server
I20260810 00:52:00.672139 10629 webserver.cc:460] Webserver started at https://127.0.0.1:35281/ using document root <none> and password file <none>
I20260810 00:52:00.672366 10629 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 00:52:00.672940 10629 webserver.cc:460] Webserver started at http://127.0.0.1:39351/ using document root <none> and password file <none>
2026-08-10T00:52:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:52:00Z Disabled control of system clock
I20260810 00:52:00.701586 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:40231
--webserver_interface=127.10.97.126
--webserver_port=0
--builtin_ntp_servers=127.10.97.84:39261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:40231
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35281/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323077328248-10629-0/testchainca.cert with env {}
W20260810 00:52:01.012092 18984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:01.012358 18984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:01.012491 18984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:01.019344 18984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:52:01.019449 18984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:01.019512 18984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:01.019600 18984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:52:01.019656 18984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 00:52:01.019738 18984 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 00:52:01.019789 18984 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35281/jwks.json
I20260810 00:52:01.029188 18984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:40231
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:40231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35281/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323077328248-10629-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:01.030272 18984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:01.031440 18984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:01.042335 18990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:01.042413 18994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:01.042371 18992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:01.043248 18984 server_base.cc:1034] running on GCE node
I20260810 00:52:01.043974 18984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:01.045195 18984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:01.046393 18984 hybrid_clock.cc:648] HybridClock initialized: now 1786323121046351 us; error 31 us; skew 500 ppm
I20260810 00:52:01.046701 18984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:52:01.048723 18984 webserver.cc:460] Webserver started at http://127.10.97.126:33581/ using document root <none> and password file <none>
I20260810 00:52:01.049327 18984 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:01.049432 18984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:01.049721 18984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:01.052407 18984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "521dd1e32bf74c9e8357bf1be5e52243"
format_stamp: "Formatted at 2026-08-10 00:52:01 on dist-test-slave-jlzn"
server_key: "d946235c76913340ce6c15f52d71aba7"
server_key_iv: "f0ca6146421727f76b4cdad28551150b"
server_key_version: "encryptionkey@0"
I20260810 00:52:01.053145 18984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "521dd1e32bf74c9e8357bf1be5e52243"
format_stamp: "Formatted at 2026-08-10 00:52:01 on dist-test-slave-jlzn"
server_key: "d946235c76913340ce6c15f52d71aba7"
server_key_iv: "f0ca6146421727f76b4cdad28551150b"
server_key_version: "encryptionkey@0"
I20260810 00:52:01.057843 18984 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:52:01.060899 19007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:01.062021 18984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:52:01.062184 18984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "521dd1e32bf74c9e8357bf1be5e52243"
format_stamp: "Formatted at 2026-08-10 00:52:01 on dist-test-slave-jlzn"
server_key: "d946235c76913340ce6c15f52d71aba7"
server_key_iv: "f0ca6146421727f76b4cdad28551150b"
server_key_version: "encryptionkey@0"
I20260810 00:52:01.062368 18984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:01.089046 18984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:01.089708 18984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:01.089969 18984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:01.105873 18984 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:40231
I20260810 00:52:01.105908 19099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:40231 every 8 connection(s)
I20260810 00:52:01.107002 18984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:52:01.110301 19100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:01.115700 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 18984
I20260810 00:52:01.115916 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:52:01.116520 10629 external_mini_cluster.cc:1428] Setting key f36c09765cbb196ae4463fdf075b818d
I20260810 00:52:01.120190 19100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243: Bootstrap starting.
I20260810 00:52:01.123268 19100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:01.124395 19100 log.cc:824] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:01.127024 19100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243: No bootstrap required, opened a new log
I20260810 00:52:01.128643 19106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:01.118192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53362 (local address 127.10.97.126:40231)
0810 00:52:01.119045 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:01.119055 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:01.119095 (+ 40us) server_negotiation.cc:407] Connection header received
0810 00:52:01.120760 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:01.120780 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:01.121204 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:01.121506 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:01.122078 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:01.123290 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:01.124621 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:01.125358 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:01.125518 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:01.126574 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:01.126584 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:01.126596 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:52:01.127069 (+ 473us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:01.127160 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:52:01.127186 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:01.127281 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:01.127504 (+ 223us) server_negotiation.cc:299] Negotiation successful
0810 00:52:01.127761 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":307,"threads_started":1}
I20260810 00:52:01.132285 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:52:01.134886 19100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521dd1e32bf74c9e8357bf1be5e52243" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 40231 } }
I20260810 00:52:01.135303 19100 raft_consensus.cc:374] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:01.135418 19100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 521dd1e32bf74c9e8357bf1be5e52243, State: Initialized, Role: FOLLOWER
I20260810 00:52:01.135943 19100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521dd1e32bf74c9e8357bf1be5e52243" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 40231 } }
I20260810 00:52:01.136157 19100 raft_consensus.cc:388] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:52:01.136250 19100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:52:01.136371 19100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:01.138623 19100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521dd1e32bf74c9e8357bf1be5e52243" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 40231 } }
I20260810 00:52:01.139025 19100 leader_election.cc:302] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 521dd1e32bf74c9e8357bf1be5e52243; no voters:
I20260810 00:52:01.139521 19100 leader_election.cc:288] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:52:01.139657 19112 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:52:01.140918 19112 raft_consensus.cc:686] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243 [term 1 LEADER]: Becoming Leader. State: Replica: 521dd1e32bf74c9e8357bf1be5e52243, State: Running, Role: LEADER
I20260810 00:52:01.141371 19112 consensus_queue.cc:237] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521dd1e32bf74c9e8357bf1be5e52243" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 40231 } }
I20260810 00:52:01.141520 19100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:52:01.144025 19113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "521dd1e32bf74c9e8357bf1be5e52243" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521dd1e32bf74c9e8357bf1be5e52243" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 40231 } } }
I20260810 00:52:01.144521 19113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:01.144341 19114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 521dd1e32bf74c9e8357bf1be5e52243. Latest consensus state: current_term: 1 leader_uuid: "521dd1e32bf74c9e8357bf1be5e52243" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521dd1e32bf74c9e8357bf1be5e52243" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 40231 } } }
I20260810 00:52:01.144657 19114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:01.148593 19120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:52:01.152338 19120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:52:01.159315 19120 catalog_manager.cc:1347] Generated new cluster ID: d8076d938d584b71aec8658a7b8be2f2
I20260810 00:52:01.159415 19120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:52:01.180505 19120 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:52:01.182196 19120 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:52:01.188585 19120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 521dd1e32bf74c9e8357bf1be5e52243: Generated new TSK 0
I20260810 00:52:01.189474 19120 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 00:52:01.228365 19148 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 00:52:01.228478 19148 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 00:52:01.229261 19106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:01.202823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53366 (local address 127.10.97.126:40231)
0810 00:52:01.202957 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:01.202962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:01.203344 (+ 382us) server_negotiation.cc:407] Connection header received
0810 00:52:01.203562 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:01.203568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:01.203632 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:01.203747 (+ 115us) server_negotiation.cc:226] Negotiated authn=JWT
0810 00:52:01.204291 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:01.204965 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:01.206929 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:01.207386 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:01.207475 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:01.208044 (+ 569us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 00:52:01.228778 (+ 20734us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35281/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 00:52:01.229051 (+ 273us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53366: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35281/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":31}
W20260810 00:52:01.229378 19106 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53366: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35281/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 00:52:01.229583 19147 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:01.202645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:40231 (local address 127.0.0.1:53366)
0810 00:52:01.203246 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:01.203264 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:01.203408 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:01.203776 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:01.203781 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:01.203794 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0810 00:52:01.204129 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:01.204143 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:01.205095 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:01.205100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:01.206798 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:01.206814 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:01.207536 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:01.207543 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:01.207741 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:01.207785 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 00:52:01.228966 (+ 21181us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35281/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 00:52:01.229205 (+ 239us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.97.126:40231: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35281/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":316,"threads_started":1}
I20260810 00:52:01.230822 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 18984
I20260810 00:52:01.231835 19091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:01.320712 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 18984
2026-08-10T00:52:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (706 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 00:52:01.376966 10629 mini_oidc.cc:152] Starting JWKS server
I20260810 00:52:01.378778 10629 webserver.cc:460] Webserver started at https://127.0.0.1:38869/ using document root <none> and password file <none>
I20260810 00:52:01.378964 10629 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 00:52:01.379544 10629 webserver.cc:460] Webserver started at http://127.0.0.1:41285/ using document root <none> and password file <none>
2026-08-10T00:52:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:52:01Z Disabled control of system clock
I20260810 00:52:01.407305 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:44315
--webserver_interface=127.10.97.126
--webserver_port=0
--builtin_ntp_servers=127.10.97.84:33081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:44315
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38869/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 00:52:01.710749 19158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:01.710997 19158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:01.711088 19158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:01.717885 19158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:52:01.717973 19158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:01.718020 19158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:01.718075 19158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:52:01.718118 19158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 00:52:01.718178 19158 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 00:52:01.718217 19158 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38869/jwks.json
I20260810 00:52:01.726759 19158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:44315
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:44315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38869/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:01.727769 19158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:01.729338 19158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:01.739874 19166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:01.739979 19165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:01.739913 19169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:01.740521 19158 server_base.cc:1034] running on GCE node
I20260810 00:52:01.741278 19158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:01.742513 19158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:01.743716 19158 hybrid_clock.cc:648] HybridClock initialized: now 1786323121743670 us; error 33 us; skew 500 ppm
I20260810 00:52:01.744025 19158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:52:01.746016 19158 webserver.cc:460] Webserver started at http://127.10.97.126:35645/ using document root <none> and password file <none>
I20260810 00:52:01.746622 19158 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:01.746732 19158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:01.747027 19158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:01.749688 19158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "db5660f1659e43448088ebf51d06587e"
format_stamp: "Formatted at 2026-08-10 00:52:01 on dist-test-slave-jlzn"
server_key: "5a063d4c93ba233d8d9f1f49373fb486"
server_key_iv: "933850c129b1d6d905ac8f6d631e966b"
server_key_version: "encryptionkey@0"
I20260810 00:52:01.750423 19158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "db5660f1659e43448088ebf51d06587e"
format_stamp: "Formatted at 2026-08-10 00:52:01 on dist-test-slave-jlzn"
server_key: "5a063d4c93ba233d8d9f1f49373fb486"
server_key_iv: "933850c129b1d6d905ac8f6d631e966b"
server_key_version: "encryptionkey@0"
I20260810 00:52:01.755306 19158 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 00:52:01.758517 19178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:01.759662 19158 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:52:01.759859 19158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "db5660f1659e43448088ebf51d06587e"
format_stamp: "Formatted at 2026-08-10 00:52:01 on dist-test-slave-jlzn"
server_key: "5a063d4c93ba233d8d9f1f49373fb486"
server_key_iv: "933850c129b1d6d905ac8f6d631e966b"
server_key_version: "encryptionkey@0"
I20260810 00:52:01.760069 19158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:01.784577 19158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:01.785290 19158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:01.785560 19158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:01.801805 19158 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:44315
I20260810 00:52:01.801822 19250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:44315 every 8 connection(s)
I20260810 00:52:01.802973 19158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:52:01.806324 19251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:01.811362 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 19158
I20260810 00:52:01.811542 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:52:01.812215 10629 external_mini_cluster.cc:1428] Setting key 702c1766b9900917a7b535631d159eac
I20260810 00:52:01.817155 19251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e: Bootstrap starting.
I20260810 00:52:01.820914 19251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:01.821846 19251 log.cc:824] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:01.824925 19251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e: No bootstrap required, opened a new log
I20260810 00:52:01.826687 19255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:01.814035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41374 (local address 127.10.97.126:44315)
0810 00:52:01.815032 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:01.815046 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:01.815092 (+ 46us) server_negotiation.cc:407] Connection header received
0810 00:52:01.817282 (+ 2190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:01.817309 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:01.817838 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:01.818236 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:01.818770 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:01.820701 (+ 1931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:01.821980 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:01.822841 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:01.822966 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:01.824064 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:01.824076 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:01.824093 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:52:01.824756 (+ 663us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:01.824853 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:52:01.824879 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:01.825007 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:01.825190 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 00:52:01.825503 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":311,"threads_started":1}
I20260810 00:52:01.830444 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:52:01.833597 19251 raft_consensus.cc:348] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5660f1659e43448088ebf51d06587e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44315 } }
I20260810 00:52:01.833923 19251 raft_consensus.cc:374] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:01.834017 19251 raft_consensus.cc:729] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db5660f1659e43448088ebf51d06587e, State: Initialized, Role: FOLLOWER
I20260810 00:52:01.834553 19251 consensus_queue.cc:260] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5660f1659e43448088ebf51d06587e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44315 } }
I20260810 00:52:01.834758 19251 raft_consensus.cc:388] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:52:01.834882 19251 raft_consensus.cc:482] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:52:01.835033 19251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:01.837249 19251 raft_consensus.cc:504] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5660f1659e43448088ebf51d06587e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44315 } }
I20260810 00:52:01.837805 19251 leader_election.cc:302] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db5660f1659e43448088ebf51d06587e; no voters:
I20260810 00:52:01.838339 19251 leader_election.cc:288] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:52:01.838443 19257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:52:01.839052 19257 raft_consensus.cc:686] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e [term 1 LEADER]: Becoming Leader. State: Replica: db5660f1659e43448088ebf51d06587e, State: Running, Role: LEADER
I20260810 00:52:01.839541 19257 consensus_queue.cc:237] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5660f1659e43448088ebf51d06587e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44315 } }
I20260810 00:52:01.840205 19251 sys_catalog.cc:564] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:52:01.842816 19262 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e [sys.catalog]: SysCatalogTable state changed. Reason: New leader db5660f1659e43448088ebf51d06587e. Latest consensus state: current_term: 1 leader_uuid: "db5660f1659e43448088ebf51d06587e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5660f1659e43448088ebf51d06587e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44315 } } }
I20260810 00:52:01.843225 19262 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:01.844166 19258 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db5660f1659e43448088ebf51d06587e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5660f1659e43448088ebf51d06587e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44315 } } }
I20260810 00:52:01.844944 19258 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:01.845314 19273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:52:01.849730 19273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:52:01.858484 19273 catalog_manager.cc:1347] Generated new cluster ID: e2205558c1614fe5afa9a4095f531344
I20260810 00:52:01.858584 19273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:52:01.885403 19273 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:52:01.886584 19273 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:52:01.892381 19273 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db5660f1659e43448088ebf51d06587e: Generated new TSK 0
I20260810 00:52:01.893069 19273 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:52:01.914938 19255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:01.907170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41378 (local address 127.10.97.126:44315)
0810 00:52:01.907351 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:01.907361 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:01.907690 (+ 329us) server_negotiation.cc:407] Connection header received
0810 00:52:01.907921 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:01.907924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:01.907990 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:01.908101 (+ 111us) server_negotiation.cc:226] Negotiated authn=JWT
0810 00:52:01.908603 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:01.909303 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:01.911259 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:01.911695 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:01.911782 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:01.912356 (+ 574us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 00:52:01.914335 (+ 1979us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38869/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 00:52:01.914650 (+ 315us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41378: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38869/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":63}
W20260810 00:52:01.915052 19255 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41378: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38869/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 00:52:01.917158 19293 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:01.907002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:44315 (local address 127.0.0.1:41378)
0810 00:52:01.907591 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:01.907610 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:01.907747 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:01.908142 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:01.908148 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:01.908161 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0810 00:52:01.908466 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:01.908478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:01.909436 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:01.909441 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:01.911126 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:01.911151 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:01.911896 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:01.911903 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:01.912082 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:01.912125 (+ 43us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 00:52:01.914525 (+ 2400us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38869/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 00:52:01.916694 (+ 2169us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.97.126:44315: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38869/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":1867,"thread_start_us":315,"threads_started":1}
I20260810 00:52:01.918831 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 19158
I20260810 00:52:01.920975 19244 generic_service.cc:196] Checking for leaks (request via RPC)
E20260810 00:52:01.927276 19295 webserver.cc:533] Webserver: sslize failed:
E20260810 00:52:01.927361 19295 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 00:52:02.008638 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 19158
2026-08-10T00:52:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (691 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T00:52:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:52:02Z Disabled control of system clock
I20260810 00:52:02.096858 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:45985
--webserver_interface=127.10.97.126
--webserver_port=0
--builtin_ntp_servers=127.10.97.84:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:45985
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323077328248-10629-0/testchainca.cert with env {}
W20260810 00:52:02.402779 19304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:02.403044 19304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:02.403174 19304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:02.410070 19304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:52:02.410169 19304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:02.410229 19304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:02.410305 19304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:52:02.410362 19304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 00:52:02.410442 19304 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 00:52:02.410488 19304 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 00:52:02.419857 19304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:45985
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:45985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323077328248-10629-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:02.420971 19304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:02.422199 19304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:02.433326 19320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:02.433295 19313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:02.433360 19310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:02.434166 19304 server_base.cc:1034] running on GCE node
I20260810 00:52:02.434891 19304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:02.436154 19304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:02.437350 19304 hybrid_clock.cc:648] HybridClock initialized: now 1786323122437306 us; error 39 us; skew 500 ppm
I20260810 00:52:02.437662 19304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:52:02.439713 19304 webserver.cc:460] Webserver started at http://127.10.97.126:35901/ using document root <none> and password file <none>
I20260810 00:52:02.440356 19304 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:02.440459 19304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:02.440737 19304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:02.443339 19304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "14d700bf53e041cfb62139e9a32f73e8"
format_stamp: "Formatted at 2026-08-10 00:52:02 on dist-test-slave-jlzn"
server_key: "a98a055687ba718a4907c7b62ef48edd"
server_key_iv: "f366ac623ca235af599303a1f1c1ebcb"
server_key_version: "encryptionkey@0"
I20260810 00:52:02.444111 19304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "14d700bf53e041cfb62139e9a32f73e8"
format_stamp: "Formatted at 2026-08-10 00:52:02 on dist-test-slave-jlzn"
server_key: "a98a055687ba718a4907c7b62ef48edd"
server_key_iv: "f366ac623ca235af599303a1f1c1ebcb"
server_key_version: "encryptionkey@0"
I20260810 00:52:02.448678 19304 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:52:02.451717 19326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:02.452924 19304 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:52:02.453083 19304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "14d700bf53e041cfb62139e9a32f73e8"
format_stamp: "Formatted at 2026-08-10 00:52:02 on dist-test-slave-jlzn"
server_key: "a98a055687ba718a4907c7b62ef48edd"
server_key_iv: "f366ac623ca235af599303a1f1c1ebcb"
server_key_version: "encryptionkey@0"
I20260810 00:52:02.453272 19304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:02.471473 19304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:02.472187 19304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:02.472426 19304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:02.488508 19304 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:45985
I20260810 00:52:02.488546 19398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:45985 every 8 connection(s)
I20260810 00:52:02.489614 19304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:52:02.492777 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 19304
I20260810 00:52:02.493010 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:52:02.493180 19399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:02.493818 10629 external_mini_cluster.cc:1428] Setting key 83a02f7cad905ba0632ded9c04dea4f7
I20260810 00:52:02.505404 19399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8: Bootstrap starting.
I20260810 00:52:02.506920 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:02.495725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56424 (local address 127.10.97.126:45985)
0810 00:52:02.496812 (+ 1087us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:02.496822 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:02.496861 (+ 39us) server_negotiation.cc:407] Connection header received
0810 00:52:02.498785 (+ 1924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:02.498805 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:02.499219 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:02.499513 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:02.500061 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:02.501244 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:02.502974 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:02.503636 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:02.503755 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:02.504904 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:02.504911 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:02.504922 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:52:02.505387 (+ 465us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:02.505458 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:52:02.505477 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:02.505582 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:02.505836 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 00:52:02.506101 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":454,"threads_started":1}
I20260810 00:52:02.508528 19399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:02.509557 19399 log.cc:824] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:02.510460 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:52:02.512156 19399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8: No bootstrap required, opened a new log
I20260810 00:52:02.519140 19399 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d700bf53e041cfb62139e9a32f73e8" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 45985 } }
I20260810 00:52:02.519519 19399 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:02.519647 19399 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14d700bf53e041cfb62139e9a32f73e8, State: Initialized, Role: FOLLOWER
I20260810 00:52:02.520187 19399 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d700bf53e041cfb62139e9a32f73e8" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 45985 } }
I20260810 00:52:02.520377 19399 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:52:02.520473 19399 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:52:02.520589 19399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:02.522433 19399 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d700bf53e041cfb62139e9a32f73e8" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 45985 } }
I20260810 00:52:02.522848 19399 leader_election.cc:302] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14d700bf53e041cfb62139e9a32f73e8; no voters:
I20260810 00:52:02.523334 19399 leader_election.cc:288] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:52:02.523456 19405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:52:02.524071 19405 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8 [term 1 LEADER]: Becoming Leader. State: Replica: 14d700bf53e041cfb62139e9a32f73e8, State: Running, Role: LEADER
I20260810 00:52:02.524542 19405 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d700bf53e041cfb62139e9a32f73e8" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 45985 } }
I20260810 00:52:02.525159 19399 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:52:02.526729 19408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14d700bf53e041cfb62139e9a32f73e8. Latest consensus state: current_term: 1 leader_uuid: "14d700bf53e041cfb62139e9a32f73e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d700bf53e041cfb62139e9a32f73e8" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 45985 } } }
I20260810 00:52:02.527034 19408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:02.527258 19406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14d700bf53e041cfb62139e9a32f73e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d700bf53e041cfb62139e9a32f73e8" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 45985 } } }
I20260810 00:52:02.528645 19406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:02.532582 19411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:52:02.537290 19411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:52:02.545099 19411 catalog_manager.cc:1347] Generated new cluster ID: 9f06abf2a0d9487bbff2eaca3c4d5179
I20260810 00:52:02.545204 19411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:52:02.564741 19411 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:52:02.566380 19411 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:52:02.580082 19411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14d700bf53e041cfb62139e9a32f73e8: Generated new TSK 0
I20260810 00:52:02.580977 19411 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:52:02.627677 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:02.599672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56428 (local address 127.10.97.126:45985)
0810 00:52:02.599884 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:02.599891 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:02.600273 (+ 382us) server_negotiation.cc:407] Connection header received
0810 00:52:02.600463 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:02.600467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:02.600544 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:02.600655 (+ 111us) server_negotiation.cc:226] Negotiated authn=JWT
0810 00:52:02.601148 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:02.601851 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:02.603862 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:02.604264 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:02.604350 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:02.604855 (+ 505us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 00:52:02.627036 (+ 22181us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 00:52:02.627348 (+ 312us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56428: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":58}
W20260810 00:52:02.627823 19403 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56428: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 00:52:02.627969 19437 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:02.599559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:45985 (local address 127.0.0.1:56428)
0810 00:52:02.600173 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:02.600192 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:02.600336 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:02.600690 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:02.600696 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:02.600709 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0810 00:52:02.601003 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:02.601014 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:02.601978 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:02.601984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:02.603710 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:02.603725 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:02.604412 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:02.604418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:02.604591 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:02.604632 (+ 41us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 00:52:02.627261 (+ 22629us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 00:52:02.627537 (+ 276us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.97.126:45985: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":338,"threads_started":1}
I20260810 00:52:02.629107 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 19304
I20260810 00:52:02.630185 19391 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:02.721076 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 19304
2026-08-10T00:52:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (692 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (1464 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (395 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:52:04 dist-test-slave-jlzn krb5kdc[19463](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:52:04 dist-test-slave-jlzn krb5kdc[19463](info): set up 2 sockets
Aug 10 00:52:04 dist-test-slave-jlzn krb5kdc[19463](info): commencing operation
krb5kdc: starting...
W20260810 00:52:05.756973 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:52:05 dist-test-slave-jlzn krb5kdc[19463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:52:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:52:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:52:05.952111 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:32805
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:40871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:32805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:06.256407 19488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:06.256662 19488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:06.256752 19488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:06.263504 19488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:52:06.263597 19488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:06.263646 19488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:06.263705 19488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:52:06.263746 19488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:52:06.272786 19488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:40871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:32805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:32805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:06.273836 19488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:06.275010 19488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:06.285630 19499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:06.285851 19502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:06.286072 19497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:06.286391 19488 server_base.cc:1034] running on GCE node
I20260810 00:52:06.287146 19488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:06.288424 19488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:06.289609 19488 hybrid_clock.cc:648] HybridClock initialized: now 1786323126289556 us; error 40 us; skew 500 ppm
I20260810 00:52:06.289925 19488 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:06 dist-test-slave-jlzn krb5kdc[19463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323126, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:06.292812 19488 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:52:06.294090 19488 webserver.cc:460] Webserver started at http://127.10.97.126:33303/ using document root <none> and password file <none>
I20260810 00:52:06.294703 19488 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:06.294819 19488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:06.295091 19488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:06.297885 19488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "f74a9a17947b41fd806612f4ad273057"
format_stamp: "Formatted at 2026-08-10 00:52:06 on dist-test-slave-jlzn"
server_key: "6e4861b0386015a30d403b4ab8cb7419"
server_key_iv: "e2df7e14dbb571b4729671cf4c0331b2"
server_key_version: "encryptionkey@0"
I20260810 00:52:06.298596 19488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "f74a9a17947b41fd806612f4ad273057"
format_stamp: "Formatted at 2026-08-10 00:52:06 on dist-test-slave-jlzn"
server_key: "6e4861b0386015a30d403b4ab8cb7419"
server_key_iv: "e2df7e14dbb571b4729671cf4c0331b2"
server_key_version: "encryptionkey@0"
I20260810 00:52:06.303223 19488 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260810 00:52:06.306327 19514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:06.307468 19488 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 00:52:06.307636 19488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "f74a9a17947b41fd806612f4ad273057"
format_stamp: "Formatted at 2026-08-10 00:52:06 on dist-test-slave-jlzn"
server_key: "6e4861b0386015a30d403b4ab8cb7419"
server_key_iv: "e2df7e14dbb571b4729671cf4c0331b2"
server_key_version: "encryptionkey@0"
I20260810 00:52:06.307833 19488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:06.326736 19488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:06.330183 19488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:06.330420 19488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:06.346184 19488 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:32805
I20260810 00:52:06.346225 19589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:32805 every 8 connection(s)
I20260810 00:52:06.347342 19488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:52:06.350832 19591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:06.356453 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 19488
I20260810 00:52:06.356657 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:52:06.357259 10629 external_mini_cluster.cc:1428] Setting key 44624b9a124a3f89276a116092e15e33
I20260810 00:52:06.361299 19591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057: Bootstrap starting.
I20260810 00:52:06.364866 19591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:06.366078 19591 log.cc:824] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:06.368947 19591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057: No bootstrap required, opened a new log
Aug 10 00:52:06 dist-test-slave-jlzn krb5kdc[19463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:52:06.377154 19591 raft_consensus.cc:348] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f74a9a17947b41fd806612f4ad273057" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 32805 } }
I20260810 00:52:06.377571 19591 raft_consensus.cc:374] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:06.377676 19591 raft_consensus.cc:729] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f74a9a17947b41fd806612f4ad273057, State: Initialized, Role: FOLLOWER
I20260810 00:52:06.378279 19591 consensus_queue.cc:260] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f74a9a17947b41fd806612f4ad273057" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 32805 } }
I20260810 00:52:06.378558 19591 raft_consensus.cc:388] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:52:06.378722 19591 raft_consensus.cc:482] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:52:06.378921 19591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:06.380172 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:06.359157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49582 (local address 127.10.97.126:32805)
0810 00:52:06.359955 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:06.359970 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:06.360018 (+ 48us) server_negotiation.cc:407] Connection header received
0810 00:52:06.362313 (+ 2295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:06.362342 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:06.362772 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:06.363225 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:06.364471 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:06.366420 (+ 1949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:06.367907 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:06.368865 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:06.369056 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:06.373195 (+ 4139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:06.373202 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:06.373219 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:06.373259 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:06.376208 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:06.376961 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:06.376966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:06.376972 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:06.377122 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:06.377520 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:06.377525 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:06.377528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:06.378061 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:06.378298 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:06.378766 (+ 468us) server_negotiation.cc:299] Negotiation successful
0810 00:52:06.379147 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":311,"threads_started":1}
I20260810 00:52:06.381198 19591 raft_consensus.cc:504] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f74a9a17947b41fd806612f4ad273057" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 32805 } }
I20260810 00:52:06.381757 19591 leader_election.cc:302] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f74a9a17947b41fd806612f4ad273057; no voters:
I20260810 00:52:06.382349 19591 leader_election.cc:288] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:52:06.382436 19598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:52:06.383549 19598 raft_consensus.cc:686] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057 [term 1 LEADER]: Becoming Leader. State: Replica: f74a9a17947b41fd806612f4ad273057, State: Running, Role: LEADER
I20260810 00:52:06.384080 19598 consensus_queue.cc:237] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f74a9a17947b41fd806612f4ad273057" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 32805 } }
I20260810 00:52:06.384807 19591 sys_catalog.cc:564] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:52:06.386502 19601 sys_catalog.cc:455] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f74a9a17947b41fd806612f4ad273057. Latest consensus state: current_term: 1 leader_uuid: "f74a9a17947b41fd806612f4ad273057" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f74a9a17947b41fd806612f4ad273057" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 32805 } } }
I20260810 00:52:06.386528 19599 sys_catalog.cc:455] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f74a9a17947b41fd806612f4ad273057" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f74a9a17947b41fd806612f4ad273057" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 32805 } } }
I20260810 00:52:06.386951 19599 sys_catalog.cc:458] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:06.386951 19601 sys_catalog.cc:458] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:06.390574 19608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:52:06.395147 19608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:52:06.403462 19608 catalog_manager.cc:1347] Generated new cluster ID: d5b83f16dbbf40f6a780d033311f4b2c
I20260810 00:52:06.403548 19608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:52:06.433856 19608 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:52:06.435024 19608 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:52:06.447606 19608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f74a9a17947b41fd806612f4ad273057: Generated new TSK 0
I20260810 00:52:06.448287 19608 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
I20260810 00:52:06.508474 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:32805
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:40871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:06.830914 19631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:06.831174 19631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:06.831307 19631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:06.838382 19631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:06.838485 19631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:06.838649 19631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:52:06.847774 19631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:40871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:32805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:06.848941 19631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:06.850142 19631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:06.862388 19641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:06.862350 19642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:06.862402 19645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:06.863162 19631 server_base.cc:1034] running on GCE node
I20260810 00:52:06.863874 19631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:06.864916 19631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:06.866096 19631 hybrid_clock.cc:648] HybridClock initialized: now 1786323126866032 us; error 52 us; skew 500 ppm
I20260810 00:52:06.866457 19631 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:06 dist-test-slave-jlzn krb5kdc[19463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323126, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:06.870188 19631 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:52:06.871465 19631 webserver.cc:460] Webserver started at http://127.10.97.65:35631/ using document root <none> and password file <none>
I20260810 00:52:06.872130 19631 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:06.872232 19631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:06.872474 19631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:06.875065 19631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "6e496c943bd14d85a3cbbfd30e181ae1"
format_stamp: "Formatted at 2026-08-10 00:52:06 on dist-test-slave-jlzn"
server_key: "da6ecc0a16b0cac9d72e25ef37148c32"
server_key_iv: "97ca277957e4092b18488cb4befc072d"
server_key_version: "encryptionkey@0"
I20260810 00:52:06.875782 19631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "6e496c943bd14d85a3cbbfd30e181ae1"
format_stamp: "Formatted at 2026-08-10 00:52:06 on dist-test-slave-jlzn"
server_key: "da6ecc0a16b0cac9d72e25ef37148c32"
server_key_iv: "97ca277957e4092b18488cb4befc072d"
server_key_version: "encryptionkey@0"
I20260810 00:52:06.880736 19631 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260810 00:52:06.883978 19654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:06.885191 19631 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:52:06.885322 19631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "6e496c943bd14d85a3cbbfd30e181ae1"
format_stamp: "Formatted at 2026-08-10 00:52:06 on dist-test-slave-jlzn"
server_key: "da6ecc0a16b0cac9d72e25ef37148c32"
server_key_iv: "97ca277957e4092b18488cb4befc072d"
server_key_version: "encryptionkey@0"
I20260810 00:52:06.885489 19631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:06.906885 19631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:06.910394 19631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:06.910643 19631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:06.911517 19631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:06.912750 19631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:06.912828 19631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:06.912912 19631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:06.912953 19631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:06.942405 19631 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:44747
I20260810 00:52:06.942462 19829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:44747 every 8 connection(s)
I20260810 00:52:06.943511 19631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:52:06.948611 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 19631
I20260810 00:52:06.948848 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:52:06.949625 10629 external_mini_cluster.cc:1428] Setting key f044e6203c9ae0e3fd040fc51d3ea618
Aug 10 00:52:06 dist-test-slave-jlzn krb5kdc[19463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323126, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:52:06.962659 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:06.945781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:57513 (local address 127.10.97.126:32805)
0810 00:52:06.946207 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:06.946212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:06.947103 (+ 891us) server_negotiation.cc:407] Connection header received
0810 00:52:06.949136 (+ 2033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:06.949141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:06.949210 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:06.949314 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:06.950944 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:06.951895 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:06.953357 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:06.953899 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:06.954015 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:06.957364 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:06.957369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:06.957372 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:06.957403 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:06.959634 (+ 2231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:06.960410 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:06.960414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:06.960416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:06.960533 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:06.960964 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:06.960966 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:06.960968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:06.961216 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:06.961324 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:06.962090 (+ 766us) server_negotiation.cc:299] Negotiation successful
0810 00:52:06.962324 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":205,"threads_started":1}
I20260810 00:52:06.963271 19837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:06.946238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:32805 (local address 127.10.97.65:57513)
0810 00:52:06.946952 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:06.946998 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:06.948855 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:06.949562 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:06.949569 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:06.950063 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:06.950761 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:06.950787 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:06.952024 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:06.952028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:06.953211 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:06.953223 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:06.954068 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:06.954072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:06.954259 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:06.954829 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:06.954849 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:06.957158 (+ 2309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:06.959765 (+ 2607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:06.959773 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:06.959786 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:06.960310 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:06.960636 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:06.960641 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:06.960645 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:06.960872 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:06.961326 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:06.961334 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:06.961705 (+ 371us) client_negotiation.cc:769] Sending connection context
0810 00:52:06.961967 (+ 262us) client_negotiation.cc:240] Negotiation successful
0810 00:52:06.962321 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":253,"threads_started":1}
I20260810 00:52:06.964627 19831 heartbeater.cc:347] Connected to a master server at 127.10.97.126:32805
I20260810 00:52:06.964888 19831 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:06.965503 19831 heartbeater.cc:511] Master 127.10.97.126:32805 requested a full tablet report, sending...
I20260810 00:52:06.967417 19536 ts_manager.cc:194] Registered new tserver with Master: 6e496c943bd14d85a3cbbfd30e181ae1 (127.10.97.65:44747)
I20260810 00:52:06.968808 19536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:57513
WARNING: no policy specified for kudu/127.10.97.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
I20260810 00:52:06.998200 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:32805
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:40871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:07.305837 19844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:07.306108 19844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:07.306242 19844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:07.313393 19844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:07.313494 19844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:07.313661 19844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:52:07.322899 19844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:40871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:32805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:07.324072 19844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:07.325278 19844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:07.337440 19852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:07.337426 19856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:07.337489 19859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:07.338196 19844 server_base.cc:1034] running on GCE node
I20260810 00:52:07.338737 19844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:07.339551 19844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:07.340739 19844 hybrid_clock.cc:648] HybridClock initialized: now 1786323127340700 us; error 25 us; skew 500 ppm
I20260810 00:52:07.341012 19844 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:07 dist-test-slave-jlzn krb5kdc[19463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323127, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:07.343945 19844 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:52:07.345171 19844 webserver.cc:460] Webserver started at http://127.10.97.66:36923/ using document root <none> and password file <none>
I20260810 00:52:07.345772 19844 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:07.345880 19844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:07.346168 19844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:07.348865 19844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "32838edc347f45f795f032cf28945bf2"
format_stamp: "Formatted at 2026-08-10 00:52:07 on dist-test-slave-jlzn"
server_key: "a8928aa7c8d1c4a7cadfbb877d032f14"
server_key_iv: "42ac28d3c3c62698456cf97abe94a9eb"
server_key_version: "encryptionkey@0"
I20260810 00:52:07.349572 19844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "32838edc347f45f795f032cf28945bf2"
format_stamp: "Formatted at 2026-08-10 00:52:07 on dist-test-slave-jlzn"
server_key: "a8928aa7c8d1c4a7cadfbb877d032f14"
server_key_iv: "42ac28d3c3c62698456cf97abe94a9eb"
server_key_version: "encryptionkey@0"
I20260810 00:52:07.354436 19844 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 00:52:07.357712 19869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:07.358955 19844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 00:52:07.359102 19844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "32838edc347f45f795f032cf28945bf2"
format_stamp: "Formatted at 2026-08-10 00:52:07 on dist-test-slave-jlzn"
server_key: "a8928aa7c8d1c4a7cadfbb877d032f14"
server_key_iv: "42ac28d3c3c62698456cf97abe94a9eb"
server_key_version: "encryptionkey@0"
I20260810 00:52:07.359294 19844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:07.381176 19844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:07.385252 19844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:07.385516 19844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:07.386616 19844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:07.387949 19844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:07.388019 19844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:07.388105 19844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:07.388154 19844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:07.416895 19844 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:42521
I20260810 00:52:07.416956 20048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:42521 every 8 connection(s)
I20260810 00:52:07.418028 19844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:52:07.424410 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 19844
I20260810 00:52:07.424634 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:52:07.425176 10629 external_mini_cluster.cc:1428] Setting key 82b8a08de2fbee8de0f591ad5729053e
Aug 10 00:52:07 dist-test-slave-jlzn krb5kdc[19463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323127, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:52:07.437379 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:07.420311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:42595 (local address 127.10.97.126:32805)
0810 00:52:07.420470 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:07.420475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:07.421548 (+ 1073us) server_negotiation.cc:407] Connection header received
0810 00:52:07.423323 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:07.423328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:07.423410 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:07.423495 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:07.425273 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:07.425997 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:07.427923 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:07.428345 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:07.428450 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:07.432014 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:07.432019 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:07.432022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:07.432054 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:07.434572 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:07.435234 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:07.435238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:07.435240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:07.435368 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:07.435769 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:07.435773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:07.435775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:07.436060 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:07.436219 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:07.436827 (+ 608us) server_negotiation.cc:299] Negotiation successful
0810 00:52:07.437045 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 00:52:07.438230 20052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:07.420721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:32805 (local address 127.10.97.66:42595)
0810 00:52:07.421398 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:07.421433 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:07.423064 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:07.423722 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:07.423729 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:07.424352 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:07.425083 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:07.425107 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:07.426145 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:07.426149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:07.427734 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:07.427749 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:07.428559 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:07.428565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:07.428834 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:07.429489 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:07.429514 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:07.431780 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:07.434697 (+ 2917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:07.434705 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:07.434719 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:07.435107 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:07.435470 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:07.435475 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:07.435479 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:07.435657 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:07.436195 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:07.436203 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:07.436487 (+ 284us) client_negotiation.cc:769] Sending connection context
0810 00:52:07.436703 (+ 216us) client_negotiation.cc:240] Negotiation successful
0810 00:52:07.437182 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":267,"threads_started":1}
I20260810 00:52:07.439496 20049 heartbeater.cc:347] Connected to a master server at 127.10.97.126:32805
I20260810 00:52:07.439851 20049 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:07.440641 20049 heartbeater.cc:511] Master 127.10.97.126:32805 requested a full tablet report, sending...
I20260810 00:52:07.442135 19536 ts_manager.cc:194] Registered new tserver with Master: 32838edc347f45f795f032cf28945bf2 (127.10.97.66:42521)
I20260810 00:52:07.443176 19536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:42595
WARNING: no policy specified for kudu/127.10.97.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
I20260810 00:52:07.473985 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:32805
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:40871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:07.781183 20055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:07.781451 20055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:07.781579 20055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:07.788535 20055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:07.788640 20055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:07.788779 20055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:52:07.798027 20055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:40871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:32805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:07.799099 20055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:07.800302 20055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:07.812445 20062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:07.812533 20060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:07.812497 20065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:07.813162 20055 server_base.cc:1034] running on GCE node
I20260810 00:52:07.813747 20055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:07.814572 20055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:07.815771 20055 hybrid_clock.cc:648] HybridClock initialized: now 1786323127815715 us; error 46 us; skew 500 ppm
I20260810 00:52:07.816082 20055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:07 dist-test-slave-jlzn krb5kdc[19463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323127, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:07.818928 20055 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:52:07.820289 20055 webserver.cc:460] Webserver started at http://127.10.97.67:33481/ using document root <none> and password file <none>
I20260810 00:52:07.820894 20055 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:07.821012 20055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:07.821290 20055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:07.823990 20055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "1769f24aff0e4845ac9f7c8c3d4ad3cb"
format_stamp: "Formatted at 2026-08-10 00:52:07 on dist-test-slave-jlzn"
server_key: "5ac0765a1bf9fc7a95e7ea0fcbe30f0e"
server_key_iv: "1aac895618563854a31a04dd9e40dea8"
server_key_version: "encryptionkey@0"
I20260810 00:52:07.824717 20055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "1769f24aff0e4845ac9f7c8c3d4ad3cb"
format_stamp: "Formatted at 2026-08-10 00:52:07 on dist-test-slave-jlzn"
server_key: "5ac0765a1bf9fc7a95e7ea0fcbe30f0e"
server_key_iv: "1aac895618563854a31a04dd9e40dea8"
server_key_version: "encryptionkey@0"
I20260810 00:52:07.829401 20055 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:52:07.832559 20077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:07.833741 20055 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:52:07.833889 20055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "1769f24aff0e4845ac9f7c8c3d4ad3cb"
format_stamp: "Formatted at 2026-08-10 00:52:07 on dist-test-slave-jlzn"
server_key: "5ac0765a1bf9fc7a95e7ea0fcbe30f0e"
server_key_iv: "1aac895618563854a31a04dd9e40dea8"
server_key_version: "encryptionkey@0"
I20260810 00:52:07.834074 20055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:07.847991 20055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:07.851368 20055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:07.851636 20055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:07.852522 20055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:07.853696 20055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:07.853773 20055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:07.853891 20055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:07.853950 20055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:07.882968 20055 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:41227
I20260810 00:52:07.883005 20244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:41227 every 8 connection(s)
I20260810 00:52:07.884227 20055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:52:07.890292 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 20055
I20260810 00:52:07.890465 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:52:07.890870 10629 external_mini_cluster.cc:1428] Setting key 70ea5c7031d3d650bfcdc025e1c92524
Aug 10 00:52:07 dist-test-slave-jlzn krb5kdc[19463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323127, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:52:07.902319 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:07.886440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:55505 (local address 127.10.97.126:32805)
0810 00:52:07.886599 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:07.886603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:07.887697 (+ 1094us) server_negotiation.cc:407] Connection header received
0810 00:52:07.889553 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:07.889559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:07.889667 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:07.889766 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:07.891257 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:07.892218 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:07.893953 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:07.894339 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:07.894447 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:07.897612 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:07.897617 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:07.897620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:07.897658 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:07.900079 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:07.900647 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:07.900650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:07.900653 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:07.900764 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:07.901077 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:07.901080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:07.901082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:07.901311 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:07.901415 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:07.901922 (+ 507us) server_negotiation.cc:299] Negotiation successful
0810 00:52:07.902078 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 00:52:07.903095 20248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:07.886830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:32805 (local address 127.10.97.67:55505)
0810 00:52:07.887536 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:07.887587 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:07.889308 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:07.890011 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:07.890019 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:07.890554 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:07.891089 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:07.891107 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:07.892362 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:07.892366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:07.893825 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:07.893837 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:07.894542 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:07.894549 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:07.894756 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:07.895308 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:07.895328 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:07.897426 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:07.900180 (+ 2754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:07.900186 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:07.900194 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:07.900562 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:07.900837 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:07.900840 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:07.900842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:07.901006 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:07.901396 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:07.901400 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:07.901654 (+ 254us) client_negotiation.cc:769] Sending connection context
0810 00:52:07.901851 (+ 197us) client_negotiation.cc:240] Negotiation successful
0810 00:52:07.902175 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":299,"threads_started":1}
I20260810 00:52:07.904249 20246 heartbeater.cc:347] Connected to a master server at 127.10.97.126:32805
I20260810 00:52:07.904511 20246 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:07.905231 20246 heartbeater.cc:511] Master 127.10.97.126:32805 requested a full tablet report, sending...
I20260810 00:52:07.906522 19536 ts_manager.cc:194] Registered new tserver with Master: 1769f24aff0e4845ac9f7c8c3d4ad3cb (127.10.97.67:41227)
I20260810 00:52:07.907277 19536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:55505
I20260810 00:52:07.917343 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/test-admin.keytab.
Aug 10 00:52:07 dist-test-slave-jlzn krb5kdc[19463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:07.946318 10629 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 00:52:07.951004 20253 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:07.948580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56840 (local address 127.10.97.65:44747)
0810 00:52:07.949139 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:07.949147 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:07.949177 (+ 30us) server_negotiation.cc:407] Connection header received
0810 00:52:07.949332 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:07.949339 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:07.949502 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:07.949638 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:07.950600 (+ 962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56840: BlockingRecv error: recv got EOF from 127.0.0.1:56840 (error 108)
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":246,"threads_started":1}
W20260810 00:52:07.951066 10629 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/krb5cc' of type FILE
Aug 10 00:52:07 dist-test-slave-jlzn krb5kdc[19463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:07.952551 10629 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 00:52:07 dist-test-slave-jlzn krb5kdc[19463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:07.953796 10629 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 00:52:07 dist-test-slave-jlzn krb5kdc[19463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:52:07.967789 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:07.955254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56844 (local address 127.10.97.65:44747)
0810 00:52:07.955427 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:07.955432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:07.955574 (+ 142us) server_negotiation.cc:407] Connection header received
0810 00:52:07.955742 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:07.955747 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:07.955825 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:07.955908 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:07.956857 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:07.957800 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:07.958959 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:07.959489 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:07.959598 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:07.962789 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:07.962798 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:07.962809 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:07.962842 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:07.965260 (+ 2418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:07.965860 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:07.965866 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:07.965869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:07.965988 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:07.966336 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:07.966340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:07.966342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:07.966667 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:07.966941 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:07.967310 (+ 369us) server_negotiation.cc:299] Negotiation successful
0810 00:52:07.967539 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 00:52:07.971302 19831 heartbeater.cc:503] Master 127.10.97.126:32805 was elected leader, sending a full tablet report...
W20260810 00:52:07.972517 20253 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:07.971080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56860 (local address 127.10.97.65:44747)
0810 00:52:07.971255 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:07.971260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:07.971354 (+ 94us) server_negotiation.cc:407] Connection header received
0810 00:52:07.971464 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:07.971468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:07.971536 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:07.971635 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:07.972318 (+ 683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56860: BlockingRecv error: recv got EOF from 127.0.0.1:56860 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260810 00:52:07.972582 10629 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/krb5cc' of type FILE
Aug 10 00:52:07 dist-test-slave-jlzn krb5kdc[19463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:07.973971 10629 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 00:52:07 dist-test-slave-jlzn krb5kdc[19463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:07.975044 10629 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 00:52:07 dist-test-slave-jlzn krb5kdc[19463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:52:07.987444 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:07.976241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56874 (local address 127.10.97.65:44747)
0810 00:52:07.976401 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:07.976406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:07.976564 (+ 158us) server_negotiation.cc:407] Connection header received
0810 00:52:07.976707 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:07.976712 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:07.976762 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:07.976834 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:07.977709 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:07.978388 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:07.979523 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:07.979956 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:07.980066 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:07.982981 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:07.982986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:07.982989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:07.983027 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:07.985212 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:07.985771 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:07.985776 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:07.985779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:07.985891 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:07.986263 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:07.986268 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:07.986272 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:07.986506 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:07.986737 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:07.987014 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 00:52:07.987200 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260810 00:52:07.990854 20253 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:07.989529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56876 (local address 127.10.97.65:44747)
0810 00:52:07.989724 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:07.989729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:07.989877 (+ 148us) server_negotiation.cc:407] Connection header received
0810 00:52:07.989966 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:07.989969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:07.990027 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:07.990095 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:07.990664 (+ 569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56876: BlockingRecv error: recv got EOF from 127.0.0.1:56876 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
W20260810 00:52:07.990971 10629 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786323077328248-10629-0/krb5kdc/krb5cc' of type FILE
Aug 10 00:52:07 dist-test-slave-jlzn krb5kdc[19463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:07.992277 10629 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 00:52:07 dist-test-slave-jlzn krb5kdc[19463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:07.993498 10629 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 00:52:07 dist-test-slave-jlzn krb5kdc[19463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:52:08.005694 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:07.994534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56888 (local address 127.10.97.65:44747)
0810 00:52:07.994690 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:07.994695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:07.994956 (+ 261us) server_negotiation.cc:407] Connection header received
0810 00:52:07.995074 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:07.995078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:07.995126 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:07.995198 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:07.996089 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:07.996739 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:07.997912 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:07.998300 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:07.998401 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:08.001350 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:08.001354 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:08.001357 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:08.001384 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:08.003415 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:08.004041 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:08.004045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:08.004047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:08.004188 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:08.004531 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:08.004534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:08.004537 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:08.004772 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:08.005053 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:08.005264 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 00:52:08.005449 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 00:52:08.006808 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 19631
I20260810 00:52:08.016711 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:08.007403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56894 (local address 127.10.97.65:44747)
0810 00:52:08.007537 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:08.007543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:08.007850 (+ 307us) server_negotiation.cc:407] Connection header received
0810 00:52:08.007979 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:08.007981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:08.008027 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:08.008093 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:08.008830 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:08.009425 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:08.010467 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:08.010895 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:08.010990 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:08.012720 (+ 1730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:08.012725 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:08.012727 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:08.012754 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:08.014797 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:08.015280 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:08.015284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:08.015286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:08.015395 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:08.015664 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:08.015667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:08.015669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:08.015914 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:08.016098 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:08.016301 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 00:52:08.016433 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 00:52:08.017155 19824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:08.096563 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 19631
I20260810 00:52:08.119102 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 19844
Aug 10 00:52:08 dist-test-slave-jlzn krb5kdc[19463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.66@KRBTEST.COM
I20260810 00:52:08.133278 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:08.119944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44110 (local address 127.10.97.66:42521)
0810 00:52:08.120470 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:08.120477 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:08.120504 (+ 27us) server_negotiation.cc:407] Connection header received
0810 00:52:08.120562 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:08.120568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:08.120782 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:08.120908 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:08.121819 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:08.122831 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:08.124076 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:08.124600 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:08.124700 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:08.128146 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:08.128156 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:08.128170 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:08.128210 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:08.130805 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:08.131416 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:08.131422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:08.131425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:08.131556 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:08.131876 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:08.131879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:08.131881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:08.132204 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:08.132457 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:08.132685 (+ 228us) server_negotiation.cc:299] Negotiation successful
0810 00:52:08.132932 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":247,"threads_started":1}
I20260810 00:52:08.134433 20041 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:08.213330 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 19844
I20260810 00:52:08.234072 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 20055
Aug 10 00:52:08 dist-test-slave-jlzn krb5kdc[19463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.67@KRBTEST.COM
I20260810 00:52:08.249075 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:08.235011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39754 (local address 127.10.97.67:41227)
0810 00:52:08.235675 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:08.235682 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:08.235709 (+ 27us) server_negotiation.cc:407] Connection header received
0810 00:52:08.235768 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:08.235774 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:08.236013 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:08.236156 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:08.237251 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:08.238338 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:08.239580 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:08.240172 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:08.240288 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:08.243901 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:08.243911 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:08.243924 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:08.243954 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:08.246359 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:08.246981 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:08.246987 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:08.246989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:08.247110 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:08.247465 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:08.247469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:08.247471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:08.247940 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:08.248253 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:08.248539 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 00:52:08.248799 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":358,"threads_started":1}
I20260810 00:52:08.250288 20236 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:08.330667 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 20055
I20260810 00:52:08.351043 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 19488
I20260810 00:52:08.352145 19583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:08.449234 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 19488
2026-08-10T00:52:08Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3873 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:52:08 dist-test-slave-jlzn krb5kdc[20291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:52:08 dist-test-slave-jlzn krb5kdc[20291](info): set up 2 sockets
Aug 10 00:52:08 dist-test-slave-jlzn krb5kdc[20291](info): commencing operation
krb5kdc: starting...
W20260810 00:52:09.637776 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:52:09 dist-test-slave-jlzn krb5kdc[20291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:52:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:52:09Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.126.keytab.
Entry for principal oryx/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.126.keytab.
I20260810 00:52:09.834685 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:39955
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.126.keytab
--principal=oryx/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:39955
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:10.141573 20308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:10.141896 20308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:10.142042 20308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:10.148908 20308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:52:10.149011 20308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:10.149084 20308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:10.149166 20308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:52:10.149228 20308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:52:10.158332 20308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:39955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.126.keytab
--principal=oryx/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:39955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:10.159492 20308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:10.160768 20308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:10.171478 20313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:10.171475 20315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:10.171579 20318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:10.172540 20308 server_base.cc:1034] running on GCE node
I20260810 00:52:10.173229 20308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:10.174437 20308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:10.175640 20308 hybrid_clock.cc:648] HybridClock initialized: now 1786323130175597 us; error 26 us; skew 500 ppm
I20260810 00:52:10.175987 20308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:10 dist-test-slave-jlzn krb5kdc[20291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323130, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:10.181674 20308 init.cc:377] Logged in from keytab as oryx/127.10.97.126@KRBTEST.COM (short username oryx)
I20260810 00:52:10.182996 20308 webserver.cc:460] Webserver started at http://127.10.97.126:38873/ using document root <none> and password file <none>
I20260810 00:52:10.183601 20308 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:10.183732 20308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:10.184042 20308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:10.186844 20308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "9c85df2445f54578af6ec069180550e6"
format_stamp: "Formatted at 2026-08-10 00:52:10 on dist-test-slave-jlzn"
server_key: "e0ee39d9f632209810877a2a08098aae"
server_key_iv: "906fde7bf08c13c5629386a453c59fe6"
server_key_version: "encryptionkey@0"
I20260810 00:52:10.187562 20308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "9c85df2445f54578af6ec069180550e6"
format_stamp: "Formatted at 2026-08-10 00:52:10 on dist-test-slave-jlzn"
server_key: "e0ee39d9f632209810877a2a08098aae"
server_key_iv: "906fde7bf08c13c5629386a453c59fe6"
server_key_version: "encryptionkey@0"
I20260810 00:52:10.192432 20308 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 00:52:10.195547 20328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:10.196730 20308 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260810 00:52:10.196879 20308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "9c85df2445f54578af6ec069180550e6"
format_stamp: "Formatted at 2026-08-10 00:52:10 on dist-test-slave-jlzn"
server_key: "e0ee39d9f632209810877a2a08098aae"
server_key_iv: "906fde7bf08c13c5629386a453c59fe6"
server_key_version: "encryptionkey@0"
I20260810 00:52:10.197049 20308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:10.228475 20308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:10.232012 20308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:10.232257 20308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:10.248802 20308 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:39955
I20260810 00:52:10.248836 20415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:39955 every 8 connection(s)
I20260810 00:52:10.250015 20308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:52:10.250406 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 20308
I20260810 00:52:10.250655 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:52:10.251389 10629 external_mini_cluster.cc:1428] Setting key cac413f3dc180ab23aad50002223a084
I20260810 00:52:10.254209 20416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 00:52:10 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM
I20260810 00:52:10.266192 20416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6: Bootstrap starting.
I20260810 00:52:10.269085 20416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:10.270016 20416 log.cc:824] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:10.272686 20419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:10.253282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35472 (local address 127.10.97.126:39955)
0810 00:52:10.254064 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:10.254080 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:10.254123 (+ 43us) server_negotiation.cc:407] Connection header received
0810 00:52:10.255665 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:10.255685 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:10.256063 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:10.256377 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:10.258148 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:10.259439 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:10.261118 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:10.261745 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:10.261860 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:10.266131 (+ 4271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:10.266139 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:10.266152 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:10.266194 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:10.268933 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:10.269593 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:10.269599 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:10.269604 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:10.269782 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:10.270129 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:10.270134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:10.270137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:10.270789 (+ 652us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:10.271113 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:10.271395 (+ 282us) server_negotiation.cc:299] Negotiation successful
0810 00:52:10.271745 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":327,"threads_started":1}
I20260810 00:52:10.272955 20416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6: No bootstrap required, opened a new log
I20260810 00:52:10.280314 20416 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } }
I20260810 00:52:10.280609 20416 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:10.280704 20416 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c85df2445f54578af6ec069180550e6, State: Initialized, Role: FOLLOWER
I20260810 00:52:10.281247 20416 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } }
I20260810 00:52:10.281462 20416 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:52:10.281581 20416 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:52:10.281735 20416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:10.283634 20416 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } }
I20260810 00:52:10.284076 20416 leader_election.cc:302] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c85df2445f54578af6ec069180550e6; no voters:
I20260810 00:52:10.284579 20416 leader_election.cc:288] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:52:10.284682 20421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:52:10.285272 20421 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [term 1 LEADER]: Becoming Leader. State: Replica: 9c85df2445f54578af6ec069180550e6, State: Running, Role: LEADER
I20260810 00:52:10.285751 20421 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } }
I20260810 00:52:10.286309 20416 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:52:10.289366 20424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c85df2445f54578af6ec069180550e6. Latest consensus state: current_term: 1 leader_uuid: "9c85df2445f54578af6ec069180550e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } }
I20260810 00:52:10.289780 20424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:10.290283 20422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c85df2445f54578af6ec069180550e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } }
I20260810 00:52:10.290620 20422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:10.292177 20433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:52:10.296057 20433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:52:10.303076 20433 catalog_manager.cc:1347] Generated new cluster ID: 94d15d9e56fe40589d69ace7b6a3507e
I20260810 00:52:10.303171 20433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:52:10.326157 20433 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:52:10.327907 20433 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:52:10.340022 20433 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6: Generated new TSK 0
I20260810 00:52:10.340958 20433 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.65.keytab.
Entry for principal oryx/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.65.keytab.
I20260810 00:52:10.397276 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:39955
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.65.keytab
--principal=oryx/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:10.707484 20451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:10.707744 20451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:10.707870 20451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:10.714672 20451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:10.714767 20451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:10.714879 20451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:52:10.723893 20451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.65.keytab
--principal=oryx/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:39955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:10.724968 20451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:10.726145 20451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:10.738268 20456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:10.738355 20457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:10.738317 20461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:10.739131 20451 server_base.cc:1034] running on GCE node
I20260810 00:52:10.739735 20451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:10.740617 20451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:10.741812 20451 hybrid_clock.cc:648] HybridClock initialized: now 1786323130741774 us; error 30 us; skew 500 ppm
I20260810 00:52:10.742120 20451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:10 dist-test-slave-jlzn krb5kdc[20291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323130, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:10.745179 20451 init.cc:377] Logged in from keytab as oryx/127.10.97.65@KRBTEST.COM (short username oryx)
I20260810 00:52:10.746445 20451 webserver.cc:460] Webserver started at http://127.10.97.65:44355/ using document root <none> and password file <none>
I20260810 00:52:10.747061 20451 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:10.747160 20451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:10.747392 20451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:10.750157 20451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "c2cc23f25d1a4f7394bd756cf079b0fc"
format_stamp: "Formatted at 2026-08-10 00:52:10 on dist-test-slave-jlzn"
server_key: "15dc6740c12f0f5b81f5fc5cf1741b6b"
server_key_iv: "908041fd8b01bf1c3908ec41c16c3fac"
server_key_version: "encryptionkey@0"
I20260810 00:52:10.750864 20451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "c2cc23f25d1a4f7394bd756cf079b0fc"
format_stamp: "Formatted at 2026-08-10 00:52:10 on dist-test-slave-jlzn"
server_key: "15dc6740c12f0f5b81f5fc5cf1741b6b"
server_key_iv: "908041fd8b01bf1c3908ec41c16c3fac"
server_key_version: "encryptionkey@0"
I20260810 00:52:10.755584 20451 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260810 00:52:10.758715 20469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:10.759958 20451 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:52:10.760114 20451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "c2cc23f25d1a4f7394bd756cf079b0fc"
format_stamp: "Formatted at 2026-08-10 00:52:10 on dist-test-slave-jlzn"
server_key: "15dc6740c12f0f5b81f5fc5cf1741b6b"
server_key_iv: "908041fd8b01bf1c3908ec41c16c3fac"
server_key_version: "encryptionkey@0"
I20260810 00:52:10.760280 20451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:10.775138 20451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:10.778623 20451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:10.778857 20451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:10.779729 20451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:10.780916 20451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:10.780989 20451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:10.781106 20451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:10.781159 20451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:10.810921 20451 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:45981
I20260810 00:52:10.810994 20625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:45981 every 8 connection(s)
I20260810 00:52:10.812183 20451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:52:10.814235 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 20451
I20260810 00:52:10.814486 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:52:10.815284 10629 external_mini_cluster.cc:1428] Setting key 3ff64d6aeb052571abdfd676db5e3141
Aug 10 00:52:10 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323130, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM
I20260810 00:52:10.831506 20630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:10.814551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:40219 (local address 127.10.97.126:39955)
0810 00:52:10.814975 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:10.814979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:10.816154 (+ 1175us) server_negotiation.cc:407] Connection header received
0810 00:52:10.817854 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:10.817860 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:10.817938 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:10.818031 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:10.819442 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:10.820219 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:10.821692 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:10.822070 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:10.822180 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:10.825691 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:10.825697 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:10.825700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:10.825731 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:10.828690 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:10.829353 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:10.829358 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:10.829361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:10.829516 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:10.829890 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:10.829894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:10.829897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:10.830234 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:10.830364 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:10.830958 (+ 594us) server_negotiation.cc:299] Negotiation successful
0810 00:52:10.831160 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":211,"threads_started":1}
I20260810 00:52:10.832240 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:10.815231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.65:40219)
0810 00:52:10.816001 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:10.816044 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:10.817607 (+ 1563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:10.818273 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:10.818280 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:10.818744 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:10.819280 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:10.819299 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:10.820369 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:10.820373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:10.821550 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:10.821562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:10.822288 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:10.822296 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:10.822519 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:10.823100 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:10.823122 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:10.825474 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:10.828813 (+ 3339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:10.828819 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:10.828830 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:10.829231 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:10.829611 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:10.829614 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:10.829616 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:10.829791 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:10.830337 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:10.830343 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:10.830634 (+ 291us) client_negotiation.cc:769] Sending connection context
0810 00:52:10.830839 (+ 205us) client_negotiation.cc:240] Negotiation successful
0810 00:52:10.831192 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":310,"threads_started":1}
I20260810 00:52:10.833619 20627 heartbeater.cc:347] Connected to a master server at 127.10.97.126:39955
I20260810 00:52:10.833917 20627 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:10.834707 20627 heartbeater.cc:511] Master 127.10.97.126:39955 requested a full tablet report, sending...
I20260810 00:52:10.837014 20358 ts_manager.cc:194] Registered new tserver with Master: c2cc23f25d1a4f7394bd756cf079b0fc (127.10.97.65:45981)
I20260810 00:52:10.838426 20358 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:40219
WARNING: no policy specified for oryx/127.10.97.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.97.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.66.keytab.
Entry for principal oryx/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.66.keytab.
I20260810 00:52:10.864403 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:39955
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.66.keytab
--principal=oryx/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:11.180444 20635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:11.180727 20635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:11.180862 20635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:11.187772 20635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:11.187887 20635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:11.188038 20635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:52:11.197512 20635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.66.keytab
--principal=oryx/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:39955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:11.198762 20635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:11.200111 20635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:11.212528 20643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:11.212528 20645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:11.212756 20649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:11.213369 20635 server_base.cc:1034] running on GCE node
I20260810 00:52:11.213912 20635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:11.214769 20635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:11.215973 20635 hybrid_clock.cc:648] HybridClock initialized: now 1786323131215923 us; error 33 us; skew 500 ppm
I20260810 00:52:11.216276 20635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:11 dist-test-slave-jlzn krb5kdc[20291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323131, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:11.219146 20635 init.cc:377] Logged in from keytab as oryx/127.10.97.66@KRBTEST.COM (short username oryx)
I20260810 00:52:11.220484 20635 webserver.cc:460] Webserver started at http://127.10.97.66:38743/ using document root <none> and password file <none>
I20260810 00:52:11.221143 20635 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:11.221264 20635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:11.221530 20635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:11.224263 20635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "81796e071d794ab0ab0e490657346658"
format_stamp: "Formatted at 2026-08-10 00:52:11 on dist-test-slave-jlzn"
server_key: "44feb90a227f106fb92037de888fe4b7"
server_key_iv: "3d910f0125c7ee730ddcd07b1a70142a"
server_key_version: "encryptionkey@0"
I20260810 00:52:11.225014 20635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "81796e071d794ab0ab0e490657346658"
format_stamp: "Formatted at 2026-08-10 00:52:11 on dist-test-slave-jlzn"
server_key: "44feb90a227f106fb92037de888fe4b7"
server_key_iv: "3d910f0125c7ee730ddcd07b1a70142a"
server_key_version: "encryptionkey@0"
I20260810 00:52:11.229707 20635 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:52:11.232851 20661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:11.234043 20635 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:52:11.234190 20635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "81796e071d794ab0ab0e490657346658"
format_stamp: "Formatted at 2026-08-10 00:52:11 on dist-test-slave-jlzn"
server_key: "44feb90a227f106fb92037de888fe4b7"
server_key_iv: "3d910f0125c7ee730ddcd07b1a70142a"
server_key_version: "encryptionkey@0"
I20260810 00:52:11.234356 20635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:11.279749 20635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:11.283349 20635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:11.283609 20635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:11.284510 20635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:11.285774 20635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:11.285852 20635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:11.285957 20635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:11.286010 20635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:11.315487 20635 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:42817
I20260810 00:52:11.315548 20841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:42817 every 8 connection(s)
I20260810 00:52:11.316617 20635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:52:11.322299 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 20635
I20260810 00:52:11.322513 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:52:11.323071 10629 external_mini_cluster.cc:1428] Setting key 6ed4932008553a45930a1df4a2a5ce9d
Aug 10 00:52:11 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323131, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM
I20260810 00:52:11.335820 20630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:11.318867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:58747 (local address 127.10.97.126:39955)
0810 00:52:11.319022 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:11.319027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:11.320051 (+ 1024us) server_negotiation.cc:407] Connection header received
0810 00:52:11.321715 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:11.321720 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:11.321787 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:11.321872 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:11.323744 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:11.324674 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:11.326279 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:11.326901 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:11.327045 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:11.330136 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:11.330142 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:11.330146 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:11.330182 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:11.333087 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:11.333719 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:11.333724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:11.333727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:11.333880 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:11.334275 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:11.334279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:11.334282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:11.334592 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:11.334724 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:11.335409 (+ 685us) server_negotiation.cc:299] Negotiation successful
0810 00:52:11.335578 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 00:52:11.337056 20844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:11.319227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.66:58747)
0810 00:52:11.319898 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:11.319943 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:11.321483 (+ 1540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:11.322100 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:11.322110 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:11.322797 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:11.323539 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:11.323564 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:11.324831 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:11.324836 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:11.326117 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:11.326140 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:11.327086 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:11.327091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:11.327288 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:11.327907 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:11.327929 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:11.329940 (+ 2011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:11.333212 (+ 3272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:11.333218 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:11.333229 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:11.333606 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:11.333982 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:11.333985 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:11.333988 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:11.334172 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:11.334698 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:11.334703 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:11.335034 (+ 331us) client_negotiation.cc:769] Sending connection context
0810 00:52:11.335325 (+ 291us) client_negotiation.cc:240] Negotiation successful
0810 00:52:11.335796 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":235,"threads_started":1}
I20260810 00:52:11.338413 20842 heartbeater.cc:347] Connected to a master server at 127.10.97.126:39955
I20260810 00:52:11.338716 20842 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:11.339498 20842 heartbeater.cc:511] Master 127.10.97.126:39955 requested a full tablet report, sending...
I20260810 00:52:11.340956 20358 ts_manager.cc:194] Registered new tserver with Master: 81796e071d794ab0ab0e490657346658 (127.10.97.66:42817)
I20260810 00:52:11.341789 20358 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:58747
WARNING: no policy specified for oryx/127.10.97.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.97.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.67.keytab.
Entry for principal oryx/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.67.keytab.
I20260810 00:52:11.372725 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:39955
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.67.keytab
--principal=oryx/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:11.680578 20852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:11.680836 20852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:11.680928 20852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:11.687778 20852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:11.687899 20852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:11.688043 20852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:52:11.697156 20852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.67.keytab
--principal=oryx/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:39955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:11.698244 20852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:11.699456 20852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:11.711575 20861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:11.711656 20864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:11.711709 20862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:11.712494 20852 server_base.cc:1034] running on GCE node
I20260810 00:52:11.713052 20852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:11.713863 20852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:11.715060 20852 hybrid_clock.cc:648] HybridClock initialized: now 1786323131715005 us; error 45 us; skew 500 ppm
I20260810 00:52:11.715345 20852 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:11 dist-test-slave-jlzn krb5kdc[20291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323131, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:11.718284 20852 init.cc:377] Logged in from keytab as oryx/127.10.97.67@KRBTEST.COM (short username oryx)
I20260810 00:52:11.719542 20852 webserver.cc:460] Webserver started at http://127.10.97.67:41117/ using document root <none> and password file <none>
I20260810 00:52:11.720196 20852 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:11.720314 20852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:11.720589 20852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:11.723300 20852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "072edf83cafd41b68224a871315bb1c6"
format_stamp: "Formatted at 2026-08-10 00:52:11 on dist-test-slave-jlzn"
server_key: "6c27a8a7657ac313809f6eebd3d74956"
server_key_iv: "0540bba0f8633357134d9ec64022729a"
server_key_version: "encryptionkey@0"
I20260810 00:52:11.724072 20852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "072edf83cafd41b68224a871315bb1c6"
format_stamp: "Formatted at 2026-08-10 00:52:11 on dist-test-slave-jlzn"
server_key: "6c27a8a7657ac313809f6eebd3d74956"
server_key_iv: "0540bba0f8633357134d9ec64022729a"
server_key_version: "encryptionkey@0"
I20260810 00:52:11.728674 20852 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 00:52:11.731786 20874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:11.733083 20852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 00:52:11.733225 20852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "072edf83cafd41b68224a871315bb1c6"
format_stamp: "Formatted at 2026-08-10 00:52:11 on dist-test-slave-jlzn"
server_key: "6c27a8a7657ac313809f6eebd3d74956"
server_key_iv: "0540bba0f8633357134d9ec64022729a"
server_key_version: "encryptionkey@0"
I20260810 00:52:11.733407 20852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:11.752090 20852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:11.755916 20852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:11.756175 20852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:11.757025 20852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:11.758255 20852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:11.758332 20852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:11.758455 20852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:11.758508 20852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:11.788162 20852 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:38697
I20260810 00:52:11.788213 21038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:38697 every 8 connection(s)
I20260810 00:52:11.789358 20852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:52:11.799256 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 20852
I20260810 00:52:11.799417 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:52:11.799875 10629 external_mini_cluster.cc:1428] Setting key 460d828d4f50e939aab544c1f9fd637c
Aug 10 00:52:11 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323131, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM
I20260810 00:52:11.807411 20630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:11.791711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:39423 (local address 127.10.97.126:39955)
0810 00:52:11.791912 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:11.791918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:11.792993 (+ 1075us) server_negotiation.cc:407] Connection header received
0810 00:52:11.794701 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:11.794705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:11.794771 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:11.794855 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:11.796202 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:11.796938 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:11.798363 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:11.798751 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:11.798833 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:11.802382 (+ 3549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:11.802388 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:11.802392 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:11.802433 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:11.804976 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:11.805585 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:11.805588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:11.805591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:11.805717 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:11.806136 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:11.806139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:11.806141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:11.806388 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:11.806495 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:11.807021 (+ 526us) server_negotiation.cc:299] Negotiation successful
0810 00:52:11.807173 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 00:52:11.808245 21041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:11.792165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.67:39423)
0810 00:52:11.792841 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:11.792886 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:11.794464 (+ 1578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:11.795021 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:11.795029 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:11.795486 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:11.796065 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:11.796084 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:11.797055 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:11.797058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:11.798258 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:11.798270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:11.798911 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:11.798915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:11.799094 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:11.799702 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:11.799729 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:11.802189 (+ 2460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:11.805097 (+ 2908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:11.805103 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:11.805112 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:11.805494 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:11.805825 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:11.805829 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:11.805832 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:11.806055 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:11.806476 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:11.806481 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:11.806752 (+ 271us) client_negotiation.cc:769] Sending connection context
0810 00:52:11.806954 (+ 202us) client_negotiation.cc:240] Negotiation successful
0810 00:52:11.807286 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":267,"threads_started":1}
I20260810 00:52:11.809357 21039 heartbeater.cc:347] Connected to a master server at 127.10.97.126:39955
I20260810 00:52:11.809592 21039 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:11.810300 21039 heartbeater.cc:511] Master 127.10.97.126:39955 requested a full tablet report, sending...
I20260810 00:52:11.811762 20358 ts_manager.cc:194] Registered new tserver with Master: 072edf83cafd41b68224a871315bb1c6 (127.10.97.67:38697)
I20260810 00:52:11.812557 20358 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:39423
I20260810 00:52:11.814997 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.10.97.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.97.125@KRBTEST.COM" created.
I20260810 00:52:11.841418 20627 heartbeater.cc:503] Master 127.10.97.126:39955 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.97.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.125.keytab.
Entry for principal oryx/127.10.97.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.125.keytab.
I20260810 00:52:11.863077 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.125:41889
--webserver_interface=127.10.97.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.125.keytab
--principal=oryx/127.10.97.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:39955,127.10.97.125:41889
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:12.169533 21046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:12.169790 21046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:12.169906 21046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:12.176973 21046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:52:12.177073 21046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:12.177155 21046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:12.177232 21046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:52:12.177289 21046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:52:12.186478 21046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:39955,127.10.97.125:41889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.125.keytab
--principal=oryx/127.10.97.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.125:41889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.97.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
W20260810 00:52:12.186816 21046 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.10.97.126:39955,127.10.97.125:41889'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 00:52:12.187821 21046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:12.189008 21046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:12.199888 21055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:12.199865 21060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:12.199822 21057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:12.200851 21046 server_base.cc:1034] running on GCE node
I20260810 00:52:12.201561 21046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:12.202821 21046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:12.204032 21046 hybrid_clock.cc:648] HybridClock initialized: now 1786323132203963 us; error 43 us; skew 500 ppm
I20260810 00:52:12.204324 21046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:12 dist-test-slave-jlzn krb5kdc[20291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323132, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:12.207281 21046 init.cc:377] Logged in from keytab as oryx/127.10.97.125@KRBTEST.COM (short username oryx)
I20260810 00:52:12.208717 21046 webserver.cc:460] Webserver started at http://127.10.97.125:35445/ using document root <none> and password file <none>
I20260810 00:52:12.209354 21046 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:12.209458 21046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:12.209748 21046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:12.212531 21046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/data/instance:
uuid: "997a5f1914d247a58a1261765413810e"
format_stamp: "Formatted at 2026-08-10 00:52:12 on dist-test-slave-jlzn"
server_key: "fa1995a771e3c1706e1e6ff825076cd4"
server_key_iv: "a1f89de15975a8529b2e007cc48c6520"
server_key_version: "encryptionkey@0"
I20260810 00:52:12.213259 21046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/wal/instance:
uuid: "997a5f1914d247a58a1261765413810e"
format_stamp: "Formatted at 2026-08-10 00:52:12 on dist-test-slave-jlzn"
server_key: "fa1995a771e3c1706e1e6ff825076cd4"
server_key_iv: "a1f89de15975a8529b2e007cc48c6520"
server_key_version: "encryptionkey@0"
I20260810 00:52:12.218024 21046 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:52:12.221102 21067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:12.222281 21046 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:52:12.222434 21046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/wal
uuid: "997a5f1914d247a58a1261765413810e"
format_stamp: "Formatted at 2026-08-10 00:52:12 on dist-test-slave-jlzn"
server_key: "fa1995a771e3c1706e1e6ff825076cd4"
server_key_iv: "a1f89de15975a8529b2e007cc48c6520"
server_key_version: "encryptionkey@0"
I20260810 00:52:12.222622 21046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:12.241004 21046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:12.244578 21046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:12.244802 21046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:12.262183 21046 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.125:41889
I20260810 00:52:12.262220 21137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.125:41889 every 8 connection(s)
I20260810 00:52:12.263388 21046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/data/info.pb
I20260810 00:52:12.266917 21138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:12.269321 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 21046
I20260810 00:52:12.274673 21138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:52:12 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.97.125@KRBTEST.COM
Aug 10 00:52:12 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323132, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.125@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM
I20260810 00:52:12.291527 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.271556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53960 (local address 127.10.97.125:41889)
0810 00:52:12.272409 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:12.272423 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:12.272475 (+ 52us) server_negotiation.cc:407] Connection header received
0810 00:52:12.274693 (+ 2218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:12.274719 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:12.275202 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:12.275618 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:12.276987 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.278492 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.280233 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.280812 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.280980 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.284761 (+ 3781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:12.284768 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:12.284778 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:12.284811 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:12.287551 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.288301 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.288307 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.288312 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.288457 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.288838 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.288841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.288843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.289356 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:12.289636 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:12.290158 (+ 522us) server_negotiation.cc:299] Negotiation successful
0810 00:52:12.290514 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":374,"threads_started":1}
I20260810 00:52:12.291791 20630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.277347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37510 (local address 127.10.97.126:39955)
0810 00:52:12.277541 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:12.277547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:12.278255 (+ 708us) server_negotiation.cc:407] Connection header received
0810 00:52:12.278420 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:12.278425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:12.278504 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:12.278620 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:12.279963 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.280741 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.282216 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.282613 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.282719 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.285954 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:12.285959 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:12.285962 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:12.285996 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:12.288968 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.289748 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.289752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.289754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.289920 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.290357 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.290362 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.290366 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.290681 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:12.290831 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:12.291261 (+ 430us) server_negotiation.cc:299] Negotiation successful
0810 00:52:12.291450 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 00:52:12.291931 21143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.277377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.0.0.1:37510)
0810 00:52:12.278110 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:12.278150 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:12.278304 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:12.278679 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:12.278685 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:12.279465 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:12.279761 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.279778 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.280880 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.280884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.282076 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.282089 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.282822 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.282829 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.282967 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.283492 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:12.283514 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:12.285778 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:12.289091 (+ 3313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.289102 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.289109 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.289640 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.290034 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.290039 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.290043 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.290257 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.290816 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:12.290823 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:12.290982 (+ 159us) client_negotiation.cc:769] Sending connection context
0810 00:52:12.291153 (+ 171us) client_negotiation.cc:240] Negotiation successful
0810 00:52:12.291449 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":363,"threads_started":1}
I20260810 00:52:12.293723 21138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:52:12 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323132, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.125@KRBTEST.COM for oryx/127.10.97.125@KRBTEST.COM
I20260810 00:52:12.304777 21143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.294511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:41889 (local address 127.0.0.1:53974)
0810 00:52:12.294658 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:12.294675 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:12.294748 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:12.295036 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:12.295039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:12.295331 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:12.295472 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.295479 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.296440 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.296444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.297259 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.297270 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.297869 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.297873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.297993 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.298444 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:12.298460 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:12.300267 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:12.302814 (+ 2547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.302817 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.302820 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.303154 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.303448 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.303450 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.303453 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.303587 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.304087 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:12.304091 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:12.304215 (+ 124us) client_negotiation.cc:769] Sending connection context
0810 00:52:12.304327 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 00:52:12.304513 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 00:52:12.304921 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.294712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53974 (local address 127.10.97.125:41889)
0810 00:52:12.294837 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:12.294841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:12.294854 (+ 13us) server_negotiation.cc:407] Connection header received
0810 00:52:12.294890 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:12.294893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:12.294939 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:12.295010 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:12.295594 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.296321 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.297372 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.297757 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.297847 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.300428 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:12.300432 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:12.300435 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:12.300463 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:12.302697 (+ 2234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.303246 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.303249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.303252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.303357 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.303670 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.303674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.303676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.303976 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:12.304090 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:12.304417 (+ 327us) server_negotiation.cc:299] Negotiation successful
0810 00:52:12.304632 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":43}
I20260810 00:52:12.308990 21138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: Bootstrap starting.
I20260810 00:52:12.311753 21138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:12.312736 21138 log.cc:824] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:12.315017 21138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: No bootstrap required, opened a new log
I20260810 00:52:12.321661 21138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } }
I20260810 00:52:12.321933 21138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:12.322022 21138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997a5f1914d247a58a1261765413810e, State: Initialized, Role: FOLLOWER
I20260810 00:52:12.322543 21138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } }
I20260810 00:52:12.323297 21145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } } }
I20260810 00:52:12.323581 21145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:52:12.324198 21138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [sys.catalog]: configured and running, proceeding with master startup.
W20260810 00:52:12.331907 21160 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 00:52:12.332012 21160 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 00:52:12.334795 21046 master_runner.cc:428] Detected that this master 997a5f1914d247a58a1261765413810e is joining an existing cluster
I20260810 00:52:12.334877 21046 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.97.125:41889
I20260810 00:52:12.340018 21145 raft_consensus.cc:482] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:52:12.340308 21145 raft_consensus.cc:504] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } }
I20260810 00:52:12.341665 21145 leader_election.cc:288] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c85df2445f54578af6ec069180550e6 (127.10.97.126:39955)
I20260810 00:52:12.342466 20378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "997a5f1914d247a58a1261765413810e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c85df2445f54578af6ec069180550e6" is_pre_election: true
I20260810 00:52:12.343678 21069 leader_election.cc:302] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 997a5f1914d247a58a1261765413810e; no voters: 9c85df2445f54578af6ec069180550e6
I20260810 00:52:12.344136 20842 heartbeater.cc:503] Master 127.10.97.126:39955 was elected leader, sending a full tablet report...
I20260810 00:52:12.344306 21145 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 00:52:12.354168 20630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.338350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37514 (local address 127.10.97.126:39955)
0810 00:52:12.338522 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:12.338527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:12.339578 (+ 1051us) server_negotiation.cc:407] Connection header received
0810 00:52:12.340017 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:12.340023 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:12.340104 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:12.340177 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:12.341716 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.342892 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.344988 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.345588 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.345692 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.348159 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:12.348165 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:12.348170 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:12.348205 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:12.351182 (+ 2977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.351971 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.351977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.351979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.352135 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.352566 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.352572 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.352575 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.352902 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:12.353024 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:12.353659 (+ 635us) server_negotiation.cc:299] Negotiation successful
0810 00:52:12.353868 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 00:52:12.355080 21170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.338694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.0.0.1:37514)
0810 00:52:12.339473 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:12.339496 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:12.339700 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:12.340395 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:12.340399 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:12.341112 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:12.341511 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.341525 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.343074 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.343080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.344811 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.344826 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.345964 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.345970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.346138 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.346772 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:12.346798 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:12.347955 (+ 1157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:12.351334 (+ 3379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.351338 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.351341 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.351835 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.352268 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.352272 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.352275 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.352434 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.353100 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:12.353105 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:12.353304 (+ 199us) client_negotiation.cc:769] Sending connection context
0810 00:52:12.354382 (+ 1078us) client_negotiation.cc:240] Negotiation successful
0810 00:52:12.354685 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"mutex_wait_us":23,"thread_start_us":380,"threads_started":1}
I20260810 00:52:12.356345 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.338470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53982 (local address 127.10.97.125:41889)
0810 00:52:12.338656 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:12.338662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:12.339947 (+ 1285us) server_negotiation.cc:407] Connection header received
0810 00:52:12.340162 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:12.340167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:12.340240 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:12.340312 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:12.341847 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.343058 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.345273 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.345871 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.345965 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.349055 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:12.349061 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:12.349066 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:12.349104 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:12.352801 (+ 3697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.353618 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.353624 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.353628 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.353779 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.354182 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.354188 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.354191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.354521 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:12.354912 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:12.355780 (+ 868us) server_negotiation.cc:299] Negotiation successful
0810 00:52:12.356007 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 00:52:12.357451 21173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.338893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:41889 (local address 127.0.0.1:53982)
0810 00:52:12.339858 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:12.339881 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:12.340027 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:12.340558 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:12.340563 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:12.341307 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:12.341671 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.341684 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.343412 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.343418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.345121 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.345145 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.346265 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.346271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.346779 (+ 508us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.347391 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:12.347414 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:12.348637 (+ 1223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:12.352937 (+ 4300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.352942 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.352945 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.353489 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.353900 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.353904 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.353908 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.354056 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.354982 (+ 926us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:12.354986 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:12.355503 (+ 517us) client_negotiation.cc:769] Sending connection context
0810 00:52:12.356721 (+ 1218us) client_negotiation.cc:240] Negotiation successful
0810 00:52:12.356993 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"mutex_wait_us":25,"thread_start_us":634,"threads_started":1}
WARNING: no policy specified for oryx/127.10.97.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.97.124@KRBTEST.COM" created.
Aug 10 00:52:12 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323130, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.126@KRBTEST.COM for oryx/127.10.97.125@KRBTEST.COM
I20260810 00:52:12.385723 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.361405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53992 (local address 127.10.97.125:41889)
0810 00:52:12.361554 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:12.361559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:12.368448 (+ 6889us) server_negotiation.cc:407] Connection header received
0810 00:52:12.368801 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:12.368807 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:12.368893 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:12.368987 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:12.370284 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.371322 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.373188 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.373896 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.374025 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.379492 (+ 5467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:12.379502 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:12.379506 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:12.379541 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:12.382879 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.383728 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.383733 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.383736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.383908 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.384366 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.384370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.384373 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.384675 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:12.384815 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:12.385297 (+ 482us) server_negotiation.cc:299] Negotiation successful
0810 00:52:12.385450 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 00:52:12.385977 21177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.361348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:41889 (local address 127.0.0.1:53992)
0810 00:52:12.368321 (+ 6973us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:12.368369 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:12.368575 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:12.369046 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:12.369054 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:12.369830 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:12.370110 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.370124 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.371479 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.371485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.373033 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.373049 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.374063 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.374070 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.374288 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.374949 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:12.374972 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:12.379277 (+ 4305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:12.383013 (+ 3736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.383026 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.383034 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.383588 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.384031 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.384036 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.384040 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.384242 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.384823 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:12.384832 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:12.385020 (+ 188us) client_negotiation.cc:769] Sending connection context
0810 00:52:12.385226 (+ 206us) client_negotiation.cc:240] Negotiation successful
0810 00:52:12.385568 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6746,"thread_start_us":6613,"threads_started":1}
I20260810 00:52:12.387797 20358 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.97.125:41889: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "997a5f1914d247a58a1261765413810e"
member_type: NON_VOTER
last_known_addr {
host: "127.10.97.125"
port: 41889
}
attrs {
promote: true
}
}
dest_uuid: "9c85df2445f54578af6ec069180550e6"
cas_config_opid_index: -1
I20260810 00:52:12.389034 20358 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: NON_VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: true } }
I20260810 00:52:12.392666 21102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:12.396313 21102 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 1 FOLLOWER]: Refusing update from remote peer 9c85df2445f54578af6ec069180550e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 00:52:12.402323 21182 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 997a5f1914d247a58a1261765413810e (127.10.97.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: NON_VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: true } } }
I20260810 00:52:12.402904 21181 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: NON_VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:52:12.406633 21182 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.97.125:41889
I20260810 00:52:12.409368 21046 master.cc:537] Master@127.10.97.125:41889 shutting down...
W20260810 00:52:12.410742 20339 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260810 00:52:12.411141 21183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9c85df2445f54578af6ec069180550e6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: NON_VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: true } } }
I20260810 00:52:12.411528 21183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: This master's current role is: LEADER
W20260810 00:52:12.414464 20339 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 -> Peer 997a5f1914d247a58a1261765413810e (127.10.97.125:41889): Couldn't send request to peer 997a5f1914d247a58a1261765413810e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.97.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.124.keytab.
Entry for principal oryx/127.10.97.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.124.keytab.
I20260810 00:52:12.417632 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.124:39429
--webserver_interface=127.10.97.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.124.keytab
--principal=oryx/127.10.97.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:39955,127.10.97.125:41889,127.10.97.124:39429
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:52:12.420559 21046 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:52:12.420883 21046 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:12.421010 21046 tablet_replica.cc:330] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: stopping tablet replica
I20260810 00:52:12.437107 21046 master.cc:552] Master@127.10.97.125:41889 shutdown complete.
I20260810 00:52:12.437268 21046 master_runner.cc:305] Clearing existing system tablet
I20260810 00:52:12.437748 21046 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:12.437852 21046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:12.440925 21046 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:52:12.442608 21193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:12.442984 21046 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:12.443298 21046 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260810 00:52:12.443389 21046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/wal
uuid: "997a5f1914d247a58a1261765413810e"
format_stamp: "Formatted at 2026-08-10 00:52:12 on dist-test-slave-jlzn"
server_key: "fa1995a771e3c1706e1e6ff825076cd4"
server_key_iv: "a1f89de15975a8529b2e007cc48c6520"
server_key_version: "encryptionkey@0"
I20260810 00:52:12.448800 21046 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:12.455492 21046 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 00:52:12.455653 21046 log.cc:1196] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 00:52:12.456086 21046 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: Deleting consensus metadata
I20260810 00:52:12.456367 21046 master_runner.cc:315] Copying system tablet from 127.10.97.126:39955
I20260810 00:52:12.461292 21046 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: tablet copy: Beginning tablet copy session from remote peer at address 127.10.97.126:39955
I20260810 00:52:12.478360 20630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.462346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37528 (local address 127.10.97.126:39955)
0810 00:52:12.462526 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:12.462532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:12.464392 (+ 1860us) server_negotiation.cc:407] Connection header received
0810 00:52:12.464594 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:12.464600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:12.464706 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:12.464814 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:12.465849 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.466954 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.468845 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.469510 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.469650 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.471941 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:12.471948 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:12.471952 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:12.471993 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:12.475460 (+ 3467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.476265 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.476270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.476274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.476431 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.476857 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.476862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.476864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.477217 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:12.477373 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:12.477812 (+ 439us) server_negotiation.cc:299] Negotiation successful
0810 00:52:12.478032 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 00:52:12.478442 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.462255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.0.0.1:37528)
0810 00:52:12.464236 (+ 1981us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:12.464261 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:12.464439 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:12.464855 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:12.464860 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:12.465379 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:12.465660 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.465673 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.467115 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.467121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.468671 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.468691 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.469678 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.469684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.469889 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.470620 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:12.470645 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:12.471707 (+ 1062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:12.475599 (+ 3892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.475605 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.475609 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.476149 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.476543 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.476548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.476552 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.476736 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.477347 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:12.477352 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:12.477529 (+ 177us) client_negotiation.cc:769] Sending connection context
0810 00:52:12.477677 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 00:52:12.477969 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1831,"thread_start_us":1697,"threads_started":1}
I20260810 00:52:12.480155 20397 tablet_copy_service.cc:140] P 9c85df2445f54578af6ec069180550e6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 997a5f1914d247a58a1261765413810e ({username='oryx', principal='oryx/127.10.97.125@KRBTEST.COM'} at 127.0.0.1:37528)
I20260810 00:52:12.480556 20397 tablet_copy_service.cc:161] P 9c85df2445f54578af6ec069180550e6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 997a5f1914d247a58a1261765413810e at {username='oryx', principal='oryx/127.10.97.125@KRBTEST.COM'} at 127.0.0.1:37528: session id = 997a5f1914d247a58a1261765413810e-00000000000000000000000000000000
I20260810 00:52:12.485178 20397 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 00:52:12.487794 21046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 00:52:12.494352 21046 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: tablet copy: Starting download of 0 data blocks...
I20260810 00:52:12.494793 21046 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: tablet copy: Starting download of 1 WAL segments...
I20260810 00:52:12.498302 21046 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 00:52:12.502665 20397 tablet_copy_service.cc:342] P 9c85df2445f54578af6ec069180550e6: Request end of tablet copy session 997a5f1914d247a58a1261765413810e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.97.125@KRBTEST.COM'} at 127.0.0.1:37528
I20260810 00:52:12.502882 20397 tablet_copy_service.cc:433] P 9c85df2445f54578af6ec069180550e6: ending tablet copy session 997a5f1914d247a58a1261765413810e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 997a5f1914d247a58a1261765413810e
I20260810 00:52:12.516238 21046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:12.518632 21205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:12.520901 21207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:12.537286 21209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:12.537628 21046 server_base.cc:1034] running on GCE node
I20260810 00:52:12.538039 21046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:12.538805 21046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:12.540030 21046 hybrid_clock.cc:648] HybridClock initialized: now 1786323132539939 us; error 52 us; skew 500 ppm
I20260810 00:52:12.540256 21046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:12 dist-test-slave-jlzn krb5kdc[20291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323132, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:12.542457 21046 init.cc:377] Logged in from keytab as oryx/127.10.97.125@KRBTEST.COM (short username oryx)
I20260810 00:52:12.550822 21046 webserver.cc:460] Webserver started at http://127.10.97.125:33925/ using document root <none> and password file <none>
I20260810 00:52:12.551216 21046 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:12.551306 21046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:12.554364 21046 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:52:12.556358 21216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:12.556770 21046 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260810 00:52:12.556890 21046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/wal
uuid: "997a5f1914d247a58a1261765413810e"
format_stamp: "Formatted at 2026-08-10 00:52:12 on dist-test-slave-jlzn"
server_key: "fa1995a771e3c1706e1e6ff825076cd4"
server_key_iv: "a1f89de15975a8529b2e007cc48c6520"
server_key_version: "encryptionkey@0"
I20260810 00:52:12.557054 21046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:12.573930 21046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:12.577798 21046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:12.602481 21046 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.125:41889
I20260810 00:52:12.602641 21283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.125:41889 every 8 connection(s)
I20260810 00:52:12.603485 21046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-1/data/info.pb
I20260810 00:52:12.608737 21285 sys_catalog.cc:263] Verifying existing consensus state
I20260810 00:52:12.611449 21285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: Bootstrap starting.
I20260810 00:52:12.625175 21285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 00:52:12.625610 21285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: Bootstrap complete.
I20260810 00:52:12.626502 21285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: NON_VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: true } }
I20260810 00:52:12.626701 21285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 997a5f1914d247a58a1261765413810e, State: Initialized, Role: LEARNER
I20260810 00:52:12.626909 21285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: NON_VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: true } }
I20260810 00:52:12.627662 21288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: NON_VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: true } } }
I20260810 00:52:12.627969 21285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:52:12.628096 21288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [sys.catalog]: This master's current role is: LEARNER
I20260810 00:52:12.641232 21310 catalog_manager.cc:1259] Loaded cluster ID: 94d15d9e56fe40589d69ace7b6a3507e
I20260810 00:52:12.641341 21310 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: loading cluster ID for follower catalog manager: success
I20260810 00:52:12.644923 21310 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: acquiring CA information for follower catalog manager: success
I20260810 00:52:12.647351 21310 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 00:52:12.776453 21190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:12.776712 21190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:12.776846 21190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:12.784093 21190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:52:12.784184 21190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:12.784260 21190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:12.784330 21190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:52:12.784391 21190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:52:12.796132 21190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:39955,127.10.97.125:41889,127.10.97.124:39429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.124.keytab
--principal=oryx/127.10.97.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.124:39429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.97.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:12.797252 21190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:12.798413 21190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:12.809221 21319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:12.809267 21320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:12.809298 21323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:12.810364 21190 server_base.cc:1034] running on GCE node
I20260810 00:52:12.811249 21190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:12.812870 21190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:12.814129 21190 hybrid_clock.cc:648] HybridClock initialized: now 1786323132814047 us; error 69 us; skew 500 ppm
I20260810 00:52:12.814414 21190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:52:12.815217 21039 heartbeater.cc:503] Master 127.10.97.126:39955 was elected leader, sending a full tablet report...
Aug 10 00:52:12 dist-test-slave-jlzn krb5kdc[20291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323132, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:12.817596 21190 init.cc:377] Logged in from keytab as oryx/127.10.97.124@KRBTEST.COM (short username oryx)
I20260810 00:52:12.819002 21190 webserver.cc:460] Webserver started at http://127.10.97.124:40213/ using document root <none> and password file <none>
I20260810 00:52:12.819631 21190 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:12.819732 21190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:12.820029 21190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:12.822763 21190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/data/instance:
uuid: "d04c2bdd7937490f9393f2c90c8fe33b"
format_stamp: "Formatted at 2026-08-10 00:52:12 on dist-test-slave-jlzn"
server_key: "87cfac57e6f547c56ad9d21c5cacbffc"
server_key_iv: "1a33cc134d2241c0fcfa087a814af9f9"
server_key_version: "encryptionkey@0"
I20260810 00:52:12.823447 21190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/wal/instance:
uuid: "d04c2bdd7937490f9393f2c90c8fe33b"
format_stamp: "Formatted at 2026-08-10 00:52:12 on dist-test-slave-jlzn"
server_key: "87cfac57e6f547c56ad9d21c5cacbffc"
server_key_iv: "1a33cc134d2241c0fcfa087a814af9f9"
server_key_version: "encryptionkey@0"
I20260810 00:52:12.828228 21190 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260810 00:52:12.831446 21332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:12.832710 21190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:52:12.832846 21190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/wal
uuid: "d04c2bdd7937490f9393f2c90c8fe33b"
format_stamp: "Formatted at 2026-08-10 00:52:12 on dist-test-slave-jlzn"
server_key: "87cfac57e6f547c56ad9d21c5cacbffc"
server_key_iv: "1a33cc134d2241c0fcfa087a814af9f9"
server_key_version: "encryptionkey@0"
I20260810 00:52:12.832996 21190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:12.863538 21190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:12.867771 21190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:12.868079 21190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:12.880677 21348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.874111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54002 (local address 127.10.97.125:41889)
0810 00:52:12.874594 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:12.874599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:12.874616 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:52:12.874677 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:12.874681 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:12.874752 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:12.874878 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:12.875377 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.876720 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.878592 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.879619 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.879763 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.879965 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:12.880322 (+ 357us) server_negotiation.cc:299] Negotiation successful
0810 00:52:12.880416 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":246,"threads_started":1}
I20260810 00:52:12.880759 21177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.873395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:41889 (local address 127.0.0.1:54002)
0810 00:52:12.873647 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:12.873668 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:12.873789 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:12.874938 (+ 1149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:12.874944 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:12.874977 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:12.875200 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.875211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.876853 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.876859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.878380 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.878397 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.879891 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.879898 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.880130 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.880200 (+ 70us) client_negotiation.cc:769] Sending connection context
0810 00:52:12.880307 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 00:52:12.880455 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136}
I20260810 00:52:12.881873 21250 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260810 00:52:12.882624 21288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c85df2445f54578af6ec069180550e6. Latest consensus state: current_term: 1 leader_uuid: "9c85df2445f54578af6ec069180550e6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: NON_VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: true } } }
I20260810 00:52:12.882874 21288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [sys.catalog]: This master's current role is: LEARNER
I20260810 00:52:12.890342 21190 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.124:39429
I20260810 00:52:12.890395 21409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.124:39429 every 8 connection(s)
I20260810 00:52:12.891556 21190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/data/info.pb
I20260810 00:52:12.894925 21410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:12.900465 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 21190
I20260810 00:52:12.901043 21410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:52:12 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323132, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.124@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM
Aug 10 00:52:12 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.97.124@KRBTEST.COM
I20260810 00:52:12.918996 20630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.903668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37530 (local address 127.10.97.126:39955)
0810 00:52:12.903873 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:12.903879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:12.904509 (+ 630us) server_negotiation.cc:407] Connection header received
0810 00:52:12.905576 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:12.905579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:12.905642 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:12.905718 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:12.907073 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.907834 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.909469 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.910063 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.910153 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.913481 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:12.913486 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:12.913489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:12.913522 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:12.916003 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.916913 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.916917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.916920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.917055 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.917477 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.917479 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.917481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.917724 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:12.917828 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:12.918489 (+ 661us) server_negotiation.cc:299] Negotiation successful
0810 00:52:12.918658 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 00:52:12.920047 21415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.903595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.0.0.1:37530)
0810 00:52:12.904356 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:12.904402 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:12.905349 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:12.905796 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:12.905804 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:12.906553 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:12.906898 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.906919 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.907971 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.907976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.909313 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.909325 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.910267 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.910275 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.910512 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.911225 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:12.911255 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:12.913314 (+ 2059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:12.916124 (+ 2810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.916136 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.916146 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.916810 (+ 664us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.917161 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.917166 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.917169 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.917383 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.917837 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:12.917844 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:12.918147 (+ 303us) client_negotiation.cc:769] Sending connection context
0810 00:52:12.918385 (+ 238us) client_negotiation.cc:240] Negotiation successful
0810 00:52:12.918812 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":3,"thread_start_us":347,"threads_started":1}
I20260810 00:52:12.920601 21414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.902600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36804 (local address 127.10.97.124:39429)
0810 00:52:12.903281 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:12.903291 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:12.903325 (+ 34us) server_negotiation.cc:407] Connection header received
0810 00:52:12.905341 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:12.905369 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:12.905650 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:12.905834 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:12.907092 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.908460 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.909809 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.910513 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.910627 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.914719 (+ 4092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:12.914727 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:12.914737 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:12.914765 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:12.917808 (+ 3043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.918420 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.918427 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.918430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.918584 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.918967 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.918972 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.918975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.919395 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:12.919665 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:12.919957 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 00:52:12.920190 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":310,"threads_started":1}
I20260810 00:52:12.921505 21410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:52:12 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323132, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.124@KRBTEST.COM for oryx/127.10.97.125@KRBTEST.COM
I20260810 00:52:12.932909 21415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.922193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:41889 (local address 127.0.0.1:54006)
0810 00:52:12.922333 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:12.922350 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:12.922436 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:12.922742 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:12.922746 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:12.923128 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:12.923248 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.923256 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.924243 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.924248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.925125 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.925138 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.925769 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.925775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.925943 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.926358 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:12.926372 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:12.928139 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:12.930720 (+ 2581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.930725 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.930729 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.931068 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.931415 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.931420 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.931424 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.931566 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.932076 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:12.932080 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:12.932214 (+ 134us) client_negotiation.cc:769] Sending connection context
0810 00:52:12.932327 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 00:52:12.932578 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 00:52:12.932971 21348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.922329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54006 (local address 127.10.97.125:41889)
0810 00:52:12.922495 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:12.922501 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:12.922517 (+ 16us) server_negotiation.cc:407] Connection header received
0810 00:52:12.922564 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:12.922567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:12.922621 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:12.922700 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:12.923380 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.924107 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.925257 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.925644 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.925747 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.928286 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:12.928291 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:12.928294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:12.928324 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:12.930602 (+ 2278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.931180 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.931185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.931187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.931315 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.931665 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.931669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.931671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.931967 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:12.932093 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:12.932441 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 00:52:12.932651 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 00:52:12.934067 21410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:52:12 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323132, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.124@KRBTEST.COM for oryx/127.10.97.124@KRBTEST.COM
I20260810 00:52:12.945591 21415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.934659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39429 (local address 127.0.0.1:36814)
0810 00:52:12.934799 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:12.934814 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:12.934898 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:12.935216 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:12.935219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:12.935535 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:12.935675 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.935682 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.936617 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.936621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.937700 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.937713 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.938470 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.938474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.938634 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.939030 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:12.939044 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:12.941012 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:12.943522 (+ 2510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.943525 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.943528 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.943917 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.944205 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.944208 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.944210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.944391 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.944813 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:12.944816 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:12.944964 (+ 148us) client_negotiation.cc:769] Sending connection context
0810 00:52:12.945111 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 00:52:12.945278 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 00:52:12.945622 21414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.934846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36814 (local address 127.10.97.124:39429)
0810 00:52:12.935003 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:12.935006 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:12.935032 (+ 26us) server_negotiation.cc:407] Connection header received
0810 00:52:12.935068 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:12.935071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:12.935117 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:12.935189 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:12.935784 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.936521 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.937813 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.938346 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.938434 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.941168 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:12.941173 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:12.941176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:12.941212 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:12.943386 (+ 2174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.944005 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.944008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.944010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.944128 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.944475 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.944478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.944479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.944723 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:12.944821 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:12.945191 (+ 370us) server_negotiation.cc:299] Negotiation successful
0810 00:52:12.945393 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":64}
I20260810 00:52:12.949734 21410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: Bootstrap starting.
I20260810 00:52:12.952557 21410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:12.953469 21410 log.cc:824] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:12.956363 21410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: No bootstrap required, opened a new log
I20260810 00:52:12.963222 21410 raft_consensus.cc:348] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } }
I20260810 00:52:12.963503 21410 raft_consensus.cc:374] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:12.963614 21410 raft_consensus.cc:729] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d04c2bdd7937490f9393f2c90c8fe33b, State: Initialized, Role: FOLLOWER
I20260810 00:52:12.964167 21410 consensus_queue.cc:260] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } }
I20260810 00:52:12.964948 21418 sys_catalog.cc:455] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } } }
I20260810 00:52:12.965309 21418 sys_catalog.cc:458] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:52:12.965812 21410 sys_catalog.cc:564] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [sys.catalog]: configured and running, proceeding with master startup.
W20260810 00:52:12.974177 21436 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 00:52:12.974296 21436 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 00:52:12.974831 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 20451
I20260810 00:52:12.977713 21190 master_runner.cc:428] Detected that this master d04c2bdd7937490f9393f2c90c8fe33b is joining an existing cluster
I20260810 00:52:12.977834 21190 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.97.124:39429
Aug 10 00:52:12 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.97.65@KRBTEST.COM
I20260810 00:52:12.996188 21439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.975864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47238 (local address 127.10.97.65:45981)
0810 00:52:12.976737 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:12.976743 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:12.976764 (+ 21us) server_negotiation.cc:407] Connection header received
0810 00:52:12.976815 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:12.976820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:12.976968 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:12.977099 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:12.978529 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.979954 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.981180 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.982025 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.982172 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.987868 (+ 5696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:12.987882 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:12.987898 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:12.987947 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:12.991774 (+ 3827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.992654 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.992661 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.992664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.992817 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.993341 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.993346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.993349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.993743 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:12.995052 (+ 1309us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:12.995221 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 00:52:12.995459 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":743,"thread_start_us":542,"threads_started":1}
I20260810 00:52:12.997747 20619 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:12.999322 20630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.981778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37546 (local address 127.10.97.126:39955)
0810 00:52:12.981943 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:12.981948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:12.983173 (+ 1225us) server_negotiation.cc:407] Connection header received
0810 00:52:12.983419 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:12.983424 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:12.983506 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:12.983588 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:12.985060 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.986146 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.988445 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.989082 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.989176 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.992270 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:12.992278 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:12.992283 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:12.992321 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:12.995492 (+ 3171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.996591 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.996596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.996599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.996782 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:12.997286 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:12.997291 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:12.997293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:12.997752 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:12.997886 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:12.998711 (+ 825us) server_negotiation.cc:299] Negotiation successful
0810 00:52:12.998936 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 00:52:13.001282 21448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.981876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.0.0.1:37546)
0810 00:52:12.983074 (+ 1198us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:12.983092 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:12.983272 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:12.983787 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:12.983791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:12.984512 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:12.984887 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.984900 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.986328 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.986334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.988004 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:12.988020 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:12.989443 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:12.989449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:12.989952 (+ 503us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:12.990894 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:12.990917 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:12.992063 (+ 1146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:12.995675 (+ 3612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.995680 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.995683 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.996451 (+ 768us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.996938 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:12.996942 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:12.996946 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:12.997151 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:12.997974 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:12.997979 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:12.998346 (+ 367us) client_negotiation.cc:769] Sending connection context
0810 00:52:12.999531 (+ 1185us) client_negotiation.cc:240] Negotiation successful
0810 00:52:12.999888 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1054,"thread_start_us":1172,"threads_started":1}
I20260810 00:52:13.017251 21348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.989084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54010 (local address 127.10.97.125:41889)
0810 00:52:12.989246 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:12.989251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:13.002055 (+ 12804us) server_negotiation.cc:407] Connection header received
0810 00:52:13.002284 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:13.002290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:13.002363 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:13.002430 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:13.004120 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.005230 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.007492 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.008197 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.008292 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:13.010749 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:13.010756 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:13.010760 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:13.010797 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:13.014183 (+ 3386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:13.014995 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:13.015000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:13.015004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:13.015151 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:13.015591 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:13.015597 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:13.015600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:13.015966 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:13.016084 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:13.016702 (+ 618us) server_negotiation.cc:299] Negotiation successful
0810 00:52:13.016924 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 00:52:13.018221 21453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.990595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:41889 (local address 127.0.0.1:54010)
0810 00:52:13.001954 (+ 11359us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:13.001972 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:13.002151 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:13.002638 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:13.002643 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:13.003364 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:13.003700 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:13.003712 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.005436 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.005441 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:13.007312 (+ 1871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:13.007332 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.008589 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.008594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:13.008755 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:13.009372 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:13.009395 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:13.010534 (+ 1139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:13.014353 (+ 3819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:13.014358 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:13.014362 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:13.014858 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:13.015299 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:13.015304 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:13.015307 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:13.015463 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:13.016210 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:13.016216 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:13.016409 (+ 193us) client_negotiation.cc:769] Sending connection context
0810 00:52:13.017472 (+ 1063us) client_negotiation.cc:240] Negotiation successful
0810 00:52:13.017782 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10569,"thread_start_us":9848,"threads_started":1}
I20260810 00:52:13.029377 21452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.982465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39429 (local address 127.0.0.1:36822)
0810 00:52:12.990391 (+ 7926us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:12.990412 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:12.991041 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:12.991510 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:12.991515 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:13.001532 (+ 10017us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:13.013721 (+ 12189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:13.013736 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.015478 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.015483 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:13.017096 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:13.017113 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.020332 (+ 3219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.020338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:13.020520 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:13.021098 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:13.021122 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:13.022184 (+ 1062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:13.026243 (+ 4059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:13.026249 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:13.026253 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:13.026793 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:13.027187 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:13.027192 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:13.027195 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:13.027342 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:13.028352 (+ 1010us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:13.028356 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:13.028533 (+ 177us) client_negotiation.cc:769] Sending connection context
0810 00:52:13.028677 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 00:52:13.028936 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7488,"thread_start_us":7612,"threads_started":1}
I20260810 00:52:13.030205 21414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:12.982860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36822 (local address 127.10.97.124:39429)
0810 00:52:12.983040 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:12.983046 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:12.990476 (+ 7430us) server_negotiation.cc:407] Connection header received
0810 00:52:12.991182 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:12.991187 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:12.991253 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:12.991324 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:13.014156 (+ 22832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.015236 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.019121 (+ 3885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.019903 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.020008 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:13.022351 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:13.022357 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:13.022362 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:13.022392 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:13.026109 (+ 3717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:13.026914 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:13.026919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:13.026923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:13.027069 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:13.027444 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:13.027449 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:13.027453 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:13.028132 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:13.028297 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:13.029670 (+ 1373us) server_negotiation.cc:299] Negotiation successful
0810 00:52:13.029876 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 00:52:13 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323130, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.126@KRBTEST.COM for oryx/127.10.97.124@KRBTEST.COM
I20260810 00:52:13.042606 21177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:13.007656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39429 (local address 127.0.0.1:36836)
0810 00:52:13.007998 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:13.008016 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:13.008120 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:13.026507 (+ 18387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:13.026513 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:13.026940 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:13.027173 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:13.027186 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.029455 (+ 2269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.029462 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:13.030857 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:13.030873 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.032038 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.032044 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:13.032219 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:13.032780 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:13.032802 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:13.035456 (+ 2654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:13.039907 (+ 4451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:13.039912 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:13.039916 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:13.040402 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:13.040852 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:13.040857 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:13.040861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:13.041027 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:13.041668 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:13.041673 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:13.041826 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 00:52:13.041970 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 00:52:13.042247 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223}
I20260810 00:52:13.043676 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:13.008301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36836 (local address 127.10.97.124:39429)
0810 00:52:13.026065 (+ 17764us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:13.026071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:13.026093 (+ 22us) server_negotiation.cc:407] Connection header received
0810 00:52:13.026171 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:13.026176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:13.026252 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:13.027324 (+ 1072us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:13.027683 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.029296 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.031156 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.031880 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.032494 (+ 614us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:13.035710 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:13.035717 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:13.035722 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:13.035756 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:13.039729 (+ 3973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:13.040560 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:13.040565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:13.040568 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:13.040719 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:13.041179 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:13.041184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:13.041188 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:13.041524 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:13.042949 (+ 1425us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:13.043077 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 00:52:13.043272 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17624,"thread_start_us":18045,"threads_started":1}
I20260810 00:52:13.044927 20358 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.97.124:39429: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b"
member_type: NON_VOTER
last_known_addr {
host: "127.10.97.124"
port: 39429
}
attrs {
promote: true
}
}
dest_uuid: "9c85df2445f54578af6ec069180550e6"
cas_config_opid_index: 5
I20260810 00:52:13.045958 20358 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: NON_VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: true } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: NON_VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: true } }
I20260810 00:52:13.048241 21250 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 1 LEARNER]: Refusing update from remote peer 9c85df2445f54578af6ec069180550e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 00:52:13.048128 21183 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d04c2bdd7937490f9393f2c90c8fe33b (127.10.97.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: NON_VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: true } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: NON_VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: true } } }
I20260810 00:52:13.048983 21375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:13.052654 21375 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [term 1 FOLLOWER]: Refusing update from remote peer 9c85df2445f54578af6ec069180550e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 00:52:13.062288 21457 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: NON_VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 00:52:13.065657 21183 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.97.124:39429
I20260810 00:52:13.066304 21190 master.cc:537] Master@127.10.97.124:39429 shutting down...
I20260810 00:52:13.067883 21183 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: NON_VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:52:13.068529 21459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9c85df2445f54578af6ec069180550e6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: NON_VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: true } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: NON_VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: true } } }
I20260810 00:52:13.070209 21459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:13.078117 21250 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d04c2bdd7937490f9393f2c90c8fe33b (127.10.97.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: NON_VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: true } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: NON_VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: true } } }
I20260810 00:52:13.082907 21288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c85df2445f54578af6ec069180550e6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: NON_VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: true } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: NON_VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: true } } }
I20260810 00:52:13.083179 21288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [sys.catalog]: This master's current role is: LEARNER
I20260810 00:52:13.083720 21183 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6: attempting to promote NON_VOTER 997a5f1914d247a58a1261765413810e to VOTER
I20260810 00:52:13.084628 21183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: NON_VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: true } }
I20260810 00:52:13.086457 21250 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 1 LEARNER]: Refusing update from remote peer 9c85df2445f54578af6ec069180550e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 00:52:13.086966 21183 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 00:52:13.089123 21459 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 997a5f1914d247a58a1261765413810e (127.10.97.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: NON_VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: true } } }
I20260810 00:52:13.089841 21250 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 997a5f1914d247a58a1261765413810e (127.10.97.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: NON_VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: true } } }
I20260810 00:52:13.093001 21457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9c85df2445f54578af6ec069180550e6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: NON_VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: true } } }
I20260810 00:52:13.093077 21183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c85df2445f54578af6ec069180550e6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: NON_VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: true } } }
I20260810 00:52:13.093457 21183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:13.093469 21457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:13.094486 21288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c85df2445f54578af6ec069180550e6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: NON_VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: true } } }
I20260810 00:52:13.094720 21288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:52:13.098065 21190 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:52:13.098440 21190 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:13.098646 21190 tablet_replica.cc:330] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: stopping tablet replica
W20260810 00:52:13.098678 20330 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 -> Peer d04c2bdd7937490f9393f2c90c8fe33b (127.10.97.124:39429): Couldn't send request to peer d04c2bdd7937490f9393f2c90c8fe33b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 00:52:13.105321 21190 master.cc:552] Master@127.10.97.124:39429 shutdown complete.
I20260810 00:52:13.105402 21190 master_runner.cc:305] Clearing existing system tablet
I20260810 00:52:13.105782 21190 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:13.105885 21190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:13.108834 21190 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260810 00:52:13.110463 21467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:13.110790 21190 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:13.111114 21190 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260810 00:52:13.111258 21190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/wal
uuid: "d04c2bdd7937490f9393f2c90c8fe33b"
format_stamp: "Formatted at 2026-08-10 00:52:12 on dist-test-slave-jlzn"
server_key: "87cfac57e6f547c56ad9d21c5cacbffc"
server_key_iv: "1a33cc134d2241c0fcfa087a814af9f9"
server_key_version: "encryptionkey@0"
I20260810 00:52:13.116051 21190 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:13.122706 21190 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 00:52:13.122906 21190 log.cc:1196] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 00:52:13.123409 21190 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: Deleting consensus metadata
I20260810 00:52:13.123689 21190 master_runner.cc:315] Copying system tablet from 127.10.97.126:39955
I20260810 00:52:13.126117 21190 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: tablet copy: Beginning tablet copy session from remote peer at address 127.10.97.126:39955
I20260810 00:52:13.129251 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 20451
I20260810 00:52:13.143604 20630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:13.127331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37560 (local address 127.10.97.126:39955)
0810 00:52:13.127798 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:13.128472 (+ 674us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:13.128492 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:52:13.128551 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:13.128556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:13.128642 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:13.128732 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:13.130753 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.131727 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.133245 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.133994 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.134101 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:13.136569 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:13.136576 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:13.136581 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:13.136620 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:13.140644 (+ 4024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:13.141488 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:13.141494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:13.141498 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:13.141656 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:13.142091 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:13.142096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:13.142099 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:13.142453 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:13.142599 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:13.143049 (+ 450us) server_negotiation.cc:299] Negotiation successful
0810 00:52:13.143266 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331}
I20260810 00:52:13.143775 21473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:13.126950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.0.0.1:37560)
0810 00:52:13.128285 (+ 1335us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:13.128304 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:13.128409 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:13.128762 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:13.128767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:13.129294 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:13.130602 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:13.130613 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.131898 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.131902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:13.133049 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:13.133068 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.134382 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.134389 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:13.134581 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:13.135270 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:13.135293 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:13.136339 (+ 1046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:13.140790 (+ 4451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:13.140796 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:13.140800 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:13.141335 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:13.141769 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:13.141774 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:13.141778 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:13.141969 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:13.142571 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:13.142581 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:13.142772 (+ 191us) client_negotiation.cc:769] Sending connection context
0810 00:52:13.142919 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 00:52:13.143245 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1215,"thread_start_us":1108,"threads_started":1}
I20260810 00:52:13.144470 20397 tablet_copy_service.cc:140] P 9c85df2445f54578af6ec069180550e6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d04c2bdd7937490f9393f2c90c8fe33b ({username='oryx', principal='oryx/127.10.97.124@KRBTEST.COM'} at 127.0.0.1:37560)
I20260810 00:52:13.144631 20397 tablet_copy_service.cc:161] P 9c85df2445f54578af6ec069180550e6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d04c2bdd7937490f9393f2c90c8fe33b at {username='oryx', principal='oryx/127.10.97.124@KRBTEST.COM'} at 127.0.0.1:37560: session id = d04c2bdd7937490f9393f2c90c8fe33b-00000000000000000000000000000000
I20260810 00:52:13.147938 20397 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 00:52:13.149703 21190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 00:52:13.154366 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:45981
--local_ip_for_outbound_sockets=127.10.97.65
--tserver_master_addrs=127.10.97.126:39955
--webserver_port=44355
--webserver_interface=127.10.97.65
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.65.keytab
--principal=oryx/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.97.126:39955,127.10.97.125:41889
--tserver_master_addrs=127.10.97.126:39955,127.10.97.125:41889,127.10.97.124:39429 with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:52:13.156522 21190 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: tablet copy: Starting download of 0 data blocks...
I20260810 00:52:13.156816 21190 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: tablet copy: Starting download of 1 WAL segments...
I20260810 00:52:13.159019 21190 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 00:52:13.162854 20397 tablet_copy_service.cc:342] P 9c85df2445f54578af6ec069180550e6: Request end of tablet copy session d04c2bdd7937490f9393f2c90c8fe33b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.97.124@KRBTEST.COM'} at 127.0.0.1:37560
I20260810 00:52:13.162987 20397 tablet_copy_service.cc:433] P 9c85df2445f54578af6ec069180550e6: ending tablet copy session d04c2bdd7937490f9393f2c90c8fe33b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d04c2bdd7937490f9393f2c90c8fe33b
I20260810 00:52:13.174813 21190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:13.177327 21475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:13.185006 21476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:13.191780 21479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:13.192970 21190 server_base.cc:1034] running on GCE node
I20260810 00:52:13.193312 21190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:13.193926 21190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:13.195122 21190 hybrid_clock.cc:648] HybridClock initialized: now 1786323133195074 us; error 33 us; skew 500 ppm
I20260810 00:52:13.195365 21190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:13 dist-test-slave-jlzn krb5kdc[20291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323133, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:13.197650 21190 init.cc:377] Logged in from keytab as oryx/127.10.97.124@KRBTEST.COM (short username oryx)
I20260810 00:52:13.198869 21190 webserver.cc:460] Webserver started at http://127.10.97.124:36401/ using document root <none> and password file <none>
I20260810 00:52:13.199210 21190 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:13.199307 21190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:13.202409 21190 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:52:13.204334 21491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:13.205163 21190 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:52:13.205292 21190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/wal
uuid: "d04c2bdd7937490f9393f2c90c8fe33b"
format_stamp: "Formatted at 2026-08-10 00:52:12 on dist-test-slave-jlzn"
server_key: "87cfac57e6f547c56ad9d21c5cacbffc"
server_key_iv: "1a33cc134d2241c0fcfa087a814af9f9"
server_key_version: "encryptionkey@0"
I20260810 00:52:13.205487 21190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:13.224382 21190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:13.228137 21190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:13.276477 21190 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.124:39429
I20260810 00:52:13.276543 21568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.124:39429 every 8 connection(s)
I20260810 00:52:13.277915 21190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/master-2/data/info.pb
I20260810 00:52:13.283185 21570 sys_catalog.cc:263] Verifying existing consensus state
I20260810 00:52:13.287029 21570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: Bootstrap starting.
I20260810 00:52:13.307590 21570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 00:52:13.308091 21570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: Bootstrap complete.
I20260810 00:52:13.309082 21570 raft_consensus.cc:348] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: NON_VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: true } }
I20260810 00:52:13.309363 21570 raft_consensus.cc:729] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d04c2bdd7937490f9393f2c90c8fe33b, State: Initialized, Role: LEARNER
I20260810 00:52:13.309620 21570 consensus_queue.cc:260] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: NON_VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: true } }
I20260810 00:52:13.310431 21577 sys_catalog.cc:455] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: NON_VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: true } } }
I20260810 00:52:13.310868 21577 sys_catalog.cc:458] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [sys.catalog]: This master's current role is: LEARNER
I20260810 00:52:13.310721 21570 sys_catalog.cc:564] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:52:13.325584 21592 catalog_manager.cc:1259] Loaded cluster ID: 94d15d9e56fe40589d69ace7b6a3507e
I20260810 00:52:13.325681 21592 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: loading cluster ID for follower catalog manager: success
I20260810 00:52:13.329310 21592 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: acquiring CA information for follower catalog manager: success
I20260810 00:52:13.331817 21592 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 00:52:13.512987 21474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:13.513228 21474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:13.513357 21474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:13.520347 21474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:13.520443 21474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:13.520604 21474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:52:13.529767 21474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.65.keytab
--principal=oryx/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:45981
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=44355
--tserver_master_addrs=127.10.97.126:39955,127.10.97.125:41889,127.10.97.124:39429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:13.530872 21474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:13.532124 21474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:13.544512 21601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:13.544539 21603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:13.544750 21605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:13.545320 21474 server_base.cc:1034] running on GCE node
I20260810 00:52:13.545857 21474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:13.546651 21474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:13.547868 21474 hybrid_clock.cc:648] HybridClock initialized: now 1786323133547811 us; error 45 us; skew 500 ppm
I20260810 00:52:13.548146 21474 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:13 dist-test-slave-jlzn krb5kdc[20291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323133, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:13.551180 21474 init.cc:377] Logged in from keytab as oryx/127.10.97.65@KRBTEST.COM (short username oryx)
I20260810 00:52:13.552510 21474 webserver.cc:460] Webserver started at http://127.10.97.65:44355/ using document root <none> and password file <none>
I20260810 00:52:13.553107 21474 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:13.553236 21474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:13.558054 21474 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260810 00:52:13.560526 21615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:13.561671 21474 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:52:13.561810 21474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "c2cc23f25d1a4f7394bd756cf079b0fc"
format_stamp: "Formatted at 2026-08-10 00:52:10 on dist-test-slave-jlzn"
server_key: "15dc6740c12f0f5b81f5fc5cf1741b6b"
server_key_iv: "908041fd8b01bf1c3908ec41c16c3fac"
server_key_version: "encryptionkey@0"
I20260810 00:52:13.562654 21474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:13.576375 21474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:13.580154 21474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:13.580394 21474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:13.581290 21474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:13.582549 21474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:13.582643 21474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:13.582747 21474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:13.582827 21474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:13.620769 21474 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:45981
I20260810 00:52:13.620858 21778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:45981 every 8 connection(s)
I20260810 00:52:13.622059 21474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:52:13.622889 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 21474
I20260810 00:52:13.623234 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 20635
I20260810 00:52:13.629428 21775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:13.616925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39429 (local address 127.0.0.1:36850)
0810 00:52:13.617367 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:13.617389 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:13.617543 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:13.620517 (+ 2974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:13.620521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:13.620552 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:13.620722 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:13.620732 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.622023 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.622027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:13.623677 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:13.623695 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.628592 (+ 4897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.628614 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:13.628826 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:13.628863 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 00:52:13.628957 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 00:52:13.629119 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":211,"threads_started":1}
I20260810 00:52:13.634351 21776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:13.617026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36850 (local address 127.10.97.124:39429)
0810 00:52:13.620155 (+ 3129us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:13.620159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:13.620176 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:52:13.620282 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:13.620285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:13.620355 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:13.620465 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:13.620892 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.621874 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.623922 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.625233 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.625367 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:13.625772 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:13.633715 (+ 7943us) server_negotiation.cc:299] Negotiation successful
0810 00:52:13.633969 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3025,"thread_start_us":2910,"threads_started":1}
I20260810 00:52:13.650997 21536 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260810 00:52:13.651777 21577 sys_catalog.cc:455] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c85df2445f54578af6ec069180550e6. Latest consensus state: current_term: 1 leader_uuid: "9c85df2445f54578af6ec069180550e6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: NON_VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: true } } }
I20260810 00:52:13.652060 21577 sys_catalog.cc:458] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [sys.catalog]: This master's current role is: LEARNER
Aug 10 00:52:13 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.97.66@KRBTEST.COM
Aug 10 00:52:13 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323133, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM
Aug 10 00:52:13 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323133, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.124@KRBTEST.COM
I20260810 00:52:13.672076 21786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:13.624784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45410 (local address 127.10.97.66:42817)
0810 00:52:13.652277 (+ 27493us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:13.652285 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:13.653145 (+ 860us) server_negotiation.cc:407] Connection header received
0810 00:52:13.653215 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:13.653221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:13.653391 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:13.653758 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:13.655181 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.656442 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.658064 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.658927 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.659059 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:13.664984 (+ 5925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:13.664998 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:13.665011 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:13.665052 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:13.669027 (+ 3975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:13.669816 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:13.669825 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:13.669829 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:13.669959 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:13.670311 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:13.670316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:13.670320 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:13.670749 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:13.671068 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:13.671385 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 00:52:13.671666 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27296,"thread_start_us":27525,"threads_started":1}
Aug 10 00:52:13 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323133, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.125@KRBTEST.COM
I20260810 00:52:13.673416 20837 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:13.681335 20630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:13.628832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:39383 (local address 127.10.97.126:39955)
0810 00:52:13.628994 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:13.629000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:13.655749 (+ 26749us) server_negotiation.cc:407] Connection header received
0810 00:52:13.657397 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:13.657402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:13.657477 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:13.657552 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:13.660052 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.661132 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.663909 (+ 2777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.664507 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.664613 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:13.672803 (+ 8190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:13.672810 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:13.672815 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:13.672855 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:13.678094 (+ 5239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:13.678896 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:13.678901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:13.678904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:13.679056 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:13.679453 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:13.679457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:13.679460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:13.679823 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:13.679972 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:13.680799 (+ 827us) server_negotiation.cc:299] Negotiation successful
0810 00:52:13.681010 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 00:52:13.682293 21787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:13.634832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:41889 (local address 127.10.97.65:35429)
0810 00:52:13.655608 (+ 20776us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:13.655627 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:13.657110 (+ 1483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:13.657972 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:13.657983 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:13.658929 (+ 946us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:13.659873 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:13.659888 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.661320 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.661325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:13.663728 (+ 2403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:13.663743 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.665453 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.665490 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:13.665776 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:13.668142 (+ 2366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:13.668165 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:13.673308 (+ 5143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:13.677866 (+ 4558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:13.677874 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:13.677888 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:13.678467 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:13.678884 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:13.678889 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:13.678891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:13.679114 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:13.679798 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:13.679825 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:13.680221 (+ 396us) client_negotiation.cc:769] Sending connection context
0810 00:52:13.680519 (+ 298us) client_negotiation.cc:240] Negotiation successful
0810 00:52:13.681000 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20619,"mutex_wait_us":291,"spinlock_wait_cycles":58880,"thread_start_us":18050,"threads_started":1}
I20260810 00:52:13.682516 21795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:13.628380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.65:39383)
0810 00:52:13.655504 (+ 27124us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:13.655519 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:13.657101 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:13.657971 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:13.657982 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:13.658802 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:13.659875 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:13.659885 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.661301 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.661305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:13.663738 (+ 2433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:13.663749 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.665562 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.665571 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:13.665758 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:13.666537 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:13.666577 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:13.672590 (+ 6013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:13.678241 (+ 5651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:13.678246 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:13.678249 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:13.678765 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:13.679182 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:13.679186 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:13.679189 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:13.679328 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:13.680029 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:13.680034 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:13.680439 (+ 405us) client_negotiation.cc:769] Sending connection context
0810 00:52:13.680586 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 00:52:13.681726 (+ 1140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26995,"mutex_wait_us":1335,"spinlock_wait_cycles":39040,"thread_start_us":22860,"threads_started":1}
I20260810 00:52:13.683537 21796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:13.635268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:35429 (local address 127.10.97.125:41889)
0810 00:52:13.653247 (+ 17979us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:13.653252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:13.655874 (+ 2622us) server_negotiation.cc:407] Connection header received
0810 00:52:13.657540 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:13.657546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:13.657619 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:13.657716 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:13.660059 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.661127 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.663957 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.664584 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.664704 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:13.673552 (+ 8848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:13.673559 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:13.673564 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:13.673604 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:13.677708 (+ 4104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:13.678612 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:13.678618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:13.678622 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:13.678764 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:13.679249 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:13.679254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:13.679257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:13.679574 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:13.679716 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:13.682970 (+ 3254us) server_negotiation.cc:299] Negotiation successful
0810 00:52:13.683204 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17889,"thread_start_us":17976,"threads_started":1}
I20260810 00:52:13.684376 21783 heartbeater.cc:347] Connected to a master server at 127.10.97.126:39955
I20260810 00:52:13.685179 21793 heartbeater.cc:347] Connected to a master server at 127.10.97.125:41889
I20260810 00:52:13.685335 21793 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:13.686296 21793 heartbeater.cc:511] Master 127.10.97.125:41889 requested a full tablet report, sending...
I20260810 00:52:13.685174 21783 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:13.687602 21783 heartbeater.cc:511] Master 127.10.97.126:39955 requested a full tablet report, sending...
I20260810 00:52:13.687767 21776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:13.625657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:60115 (local address 127.10.97.124:39429)
0810 00:52:13.634576 (+ 8919us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:13.634582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:13.653762 (+ 19180us) server_negotiation.cc:407] Connection header received
0810 00:52:13.657408 (+ 3646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:13.657414 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:13.657480 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:13.657542 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:13.660071 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.661106 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.663965 (+ 2859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.664570 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.664649 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:13.677197 (+ 12548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:13.677205 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:13.677210 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:13.677253 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:13.683555 (+ 6302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:13.685050 (+ 1495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:13.685056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:13.685060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:13.685213 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:13.685870 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:13.685876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:13.685879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:13.686223 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:13.686406 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:13.687042 (+ 636us) server_negotiation.cc:299] Negotiation successful
0810 00:52:13.687452 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8818,"thread_start_us":24528,"threads_started":1}
I20260810 00:52:13.688138 21794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:13.625139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39429 (local address 127.10.97.65:60115)
0810 00:52:13.651167 (+ 26028us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:13.653351 (+ 2184us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:13.657097 (+ 3746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:13.657972 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:13.657982 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:13.658818 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:13.659856 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:13.659882 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.661298 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.661303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:13.663724 (+ 2421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:13.663738 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:13.664927 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:13.664934 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:13.665236 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:13.667336 (+ 2100us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:13.667361 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:13.676946 (+ 9585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:13.684286 (+ 7340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:13.684291 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:13.684296 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:13.684892 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:13.685539 (+ 647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:13.685544 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:13.685547 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:13.685730 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:13.686471 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:13.686476 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:13.686633 (+ 157us) client_negotiation.cc:769] Sending connection context
0810 00:52:13.686809 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 00:52:13.687076 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25672,"mutex_wait_us":328,"spinlock_wait_cycles":1408,"thread_start_us":27714,"threads_started":1}
I20260810 00:52:13.688961 20358 ts_manager.cc:194] Re-registered known tserver with Master: c2cc23f25d1a4f7394bd756cf079b0fc (127.10.97.65:45981)
I20260810 00:52:13.689706 20358 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:39383
I20260810 00:52:13.689666 21236 ts_manager.cc:194] Registered new tserver with Master: c2cc23f25d1a4f7394bd756cf079b0fc (127.10.97.65:45981)
I20260810 00:52:13.691565 21780 heartbeater.cc:347] Connected to a master server at 127.10.97.124:39429
I20260810 00:52:13.691718 21780 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:13.692080 21780 heartbeater.cc:511] Master 127.10.97.124:39429 requested a full tablet report, sending...
I20260810 00:52:13.693799 21516 ts_manager.cc:194] Registered new tserver with Master: c2cc23f25d1a4f7394bd756cf079b0fc (127.10.97.65:45981)
I20260810 00:52:13.784102 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 20635
I20260810 00:52:13.809473 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:42817
--local_ip_for_outbound_sockets=127.10.97.66
--tserver_master_addrs=127.10.97.126:39955
--webserver_port=38743
--webserver_interface=127.10.97.66
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.66.keytab
--principal=oryx/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.97.126:39955,127.10.97.125:41889
--tserver_master_addrs=127.10.97.126:39955,127.10.97.125:41889,127.10.97.124:39429 with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:14.129719 21799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:14.129967 21799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:14.130079 21799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:14.137328 21799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:14.137426 21799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:14.137584 21799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:52:14.146749 21799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.66.keytab
--principal=oryx/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:42817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=38743
--tserver_master_addrs=127.10.97.126:39955,127.10.97.125:41889,127.10.97.124:39429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:14.147883 21799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:14.149065 21799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:14.161087 21809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:14.161120 21810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:14.161365 21812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:14.161791 21799 server_base.cc:1034] running on GCE node
I20260810 00:52:14.162343 21799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:14.163146 21799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:14.164325 21799 hybrid_clock.cc:648] HybridClock initialized: now 1786323134164278 us; error 44 us; skew 500 ppm
I20260810 00:52:14.164588 21799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:14 dist-test-slave-jlzn krb5kdc[20291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323134, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:14.167876 21799 init.cc:377] Logged in from keytab as oryx/127.10.97.66@KRBTEST.COM (short username oryx)
I20260810 00:52:14.169121 21799 webserver.cc:460] Webserver started at http://127.10.97.66:38743/ using document root <none> and password file <none>
I20260810 00:52:14.169709 21799 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:14.169816 21799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:14.174839 21799 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 00:52:14.177340 21826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:14.178546 21799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 00:52:14.178702 21799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "81796e071d794ab0ab0e490657346658"
format_stamp: "Formatted at 2026-08-10 00:52:11 on dist-test-slave-jlzn"
server_key: "44feb90a227f106fb92037de888fe4b7"
server_key_iv: "3d910f0125c7ee730ddcd07b1a70142a"
server_key_version: "encryptionkey@0"
I20260810 00:52:14.179625 21799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:14.204937 21799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:14.208931 21799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:14.209203 21799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:14.210088 21799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:14.211391 21799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:14.211488 21799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:14.211607 21799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:14.211684 21799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:14.219507 21840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c85df2445f54578af6ec069180550e6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: NON_VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: true } } }
I20260810 00:52:14.219832 21840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:14.224587 21874 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6: attempting to promote NON_VOTER d04c2bdd7937490f9393f2c90c8fe33b to VOTER
I20260810 00:52:14.225361 21874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: false } }
I20260810 00:52:14.227669 21536 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [term 1 LEARNER]: Refusing update from remote peer 9c85df2445f54578af6ec069180550e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 00:52:14.227734 21250 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 1 FOLLOWER]: Refusing update from remote peer 9c85df2445f54578af6ec069180550e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 00:52:14.228416 21877 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 00:52:14.228873 21840 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 00:52:14.231969 21877 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d04c2bdd7937490f9393f2c90c8fe33b (127.10.97.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: false } } }
I20260810 00:52:14.233233 21250 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d04c2bdd7937490f9393f2c90c8fe33b (127.10.97.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: false } } }
I20260810 00:52:14.235023 21536 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d04c2bdd7937490f9393f2c90c8fe33b (127.10.97.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: false } } }
I20260810 00:52:14.236032 21874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9c85df2445f54578af6ec069180550e6" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: false } } }
I20260810 00:52:14.236279 21874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:14.236644 21874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c85df2445f54578af6ec069180550e6" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: false } } }
I20260810 00:52:14.236903 21874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:14.237231 21874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c85df2445f54578af6ec069180550e6" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: false } } }
I20260810 00:52:14.237437 21874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:14.237993 21925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c85df2445f54578af6ec069180550e6" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: false } } }
I20260810 00:52:14.238504 21925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:52:14.239701 21928 sys_catalog.cc:455] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c85df2445f54578af6ec069180550e6" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: false } } }
I20260810 00:52:14.240273 21928 sys_catalog.cc:458] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:52:14.257232 21799 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:42817
I20260810 00:52:14.257302 22020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:42817 every 8 connection(s)
I20260810 00:52:14.258599 21799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:52:14.261437 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 21799
I20260810 00:52:14.261720 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 20852
Aug 10 00:52:14 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323134, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.125@KRBTEST.COM
Aug 10 00:52:14 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323134, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM
Aug 10 00:52:14 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323134, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.124@KRBTEST.COM
I20260810 00:52:14.292665 22034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.262636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:40425 (local address 127.10.97.125:41889)
0810 00:52:14.269301 (+ 6665us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:14.269307 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:14.273395 (+ 4088us) server_negotiation.cc:407] Connection header received
0810 00:52:14.273452 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:14.273457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:14.273543 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:14.273629 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:14.276297 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.277430 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.279891 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.280535 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.280622 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.285578 (+ 4956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:14.285584 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:14.285589 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:14.285626 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:14.289145 (+ 3519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.289992 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.289997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.290001 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.290156 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.290678 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.290684 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.290687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.291018 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:14.291141 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:14.292094 (+ 953us) server_negotiation.cc:299] Negotiation successful
0810 00:52:14.292298 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6546,"thread_start_us":6405,"threads_started":1}
I20260810 00:52:14.293833 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.261595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:50535 (local address 127.10.97.126:39955)
0810 00:52:14.276270 (+ 14675us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:14.276276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:14.276302 (+ 26us) server_negotiation.cc:407] Connection header received
0810 00:52:14.276362 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:14.276367 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:14.276442 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:14.278506 (+ 2064us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:14.278689 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.279742 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.281735 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.282452 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.282628 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.287024 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:14.287031 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:14.287035 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:14.287070 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:14.290775 (+ 3705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.291641 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.291646 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.291649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.291827 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.292275 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.292280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.292283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.292605 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:14.292726 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:14.293276 (+ 550us) server_negotiation.cc:299] Negotiation successful
0810 00:52:14.293482 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14517,"thread_start_us":14400,"threads_started":1}
I20260810 00:52:14.294871 22031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.263498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39429 (local address 127.10.97.66:40867)
0810 00:52:14.274105 (+ 10607us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:14.274122 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:14.274335 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:14.277194 (+ 2859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:14.277203 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:14.277896 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:14.278297 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:14.278309 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.279870 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.279875 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:14.281549 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:14.281564 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.282642 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.282649 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:14.283268 (+ 619us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.283941 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:14.283961 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:14.286810 (+ 2849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:14.290963 (+ 4153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:14.290967 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:14.290970 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:14.291835 (+ 865us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:14.292250 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:14.292255 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:14.292257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:14.292404 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:14.293335 (+ 931us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:14.293339 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:14.293484 (+ 145us) client_negotiation.cc:769] Sending connection context
0810 00:52:14.293630 (+ 146us) client_negotiation.cc:240] Negotiation successful
0810 00:52:14.294368 (+ 738us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10499,"mutex_wait_us":20,"thread_start_us":5966,"threads_started":1}
I20260810 00:52:14.294924 22040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.261845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:41889 (local address 127.10.97.66:40425)
0810 00:52:14.269886 (+ 8041us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:14.269935 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:14.272412 (+ 2477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:14.274689 (+ 2277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:14.274701 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:14.275358 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:14.276103 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:14.276128 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.277650 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.277658 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:14.279698 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:14.279712 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.280957 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.280963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:14.281224 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.281955 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:14.281982 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:14.285348 (+ 3366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:14.289312 (+ 3964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:14.289320 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:14.289334 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:14.289869 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:14.290307 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:14.290312 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:14.290316 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:14.290543 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:14.291233 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:14.291241 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:14.291668 (+ 427us) client_negotiation.cc:769] Sending connection context
0810 00:52:14.292935 (+ 1267us) client_negotiation.cc:240] Negotiation successful
0810 00:52:14.293425 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7734,"mutex_wait_us":1,"thread_start_us":11888,"threads_started":1}
I20260810 00:52:14.294947 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.262180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.66:50535)
0810 00:52:14.273163 (+ 10983us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:14.273181 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:14.273292 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:14.277427 (+ 4135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:14.277432 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:14.278022 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:14.278379 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:14.278391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.279894 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.279898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:14.281594 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:14.281604 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.282787 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.282793 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:14.282919 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.283573 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:14.283595 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:14.286839 (+ 3244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:14.290923 (+ 4084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:14.290928 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:14.290932 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:14.291533 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:14.291981 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:14.291985 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:14.291988 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:14.292154 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:14.292804 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:14.292809 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:14.292946 (+ 137us) client_negotiation.cc:769] Sending connection context
0810 00:52:14.293100 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 00:52:14.294484 (+ 1384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10856,"mutex_wait_us":1181,"spinlock_wait_cycles":17280,"thread_start_us":10666,"threads_started":1}
I20260810 00:52:14.296693 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.261330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:40867 (local address 127.10.97.124:39429)
0810 00:52:14.276903 (+ 15573us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:14.276908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:14.276921 (+ 13us) server_negotiation.cc:407] Connection header received
0810 00:52:14.276967 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:14.276970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:14.277050 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:14.277152 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:14.278504 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.279684 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.281753 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.282408 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.283574 (+ 1166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.287063 (+ 3489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:14.287070 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:14.287075 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:14.287108 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:14.290845 (+ 3737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.291962 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.291967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.291970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.292124 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.292547 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.292552 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.292554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.293207 (+ 653us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:14.295562 (+ 2355us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:14.295725 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 00:52:14.296204 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15475,"thread_start_us":15325,"threads_started":1}
I20260810 00:52:14.296952 22023 heartbeater.cc:347] Connected to a master server at 127.10.97.125:41889
I20260810 00:52:14.296979 22022 heartbeater.cc:347] Connected to a master server at 127.10.97.126:39955
Aug 10 00:52:14 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.97.67@KRBTEST.COM
I20260810 00:52:14.297260 22022 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:14.297427 22021 heartbeater.cc:347] Connected to a master server at 127.10.97.124:39429
I20260810 00:52:14.297549 22021 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:14.298148 22021 heartbeater.cc:511] Master 127.10.97.124:39429 requested a full tablet report, sending...
I20260810 00:52:14.298169 22022 heartbeater.cc:511] Master 127.10.97.126:39955 requested a full tablet report, sending...
I20260810 00:52:14.297266 22023 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:14.299355 22023 heartbeater.cc:511] Master 127.10.97.125:41889 requested a full tablet report, sending...
I20260810 00:52:14.299626 20358 ts_manager.cc:194] Re-registered known tserver with Master: 81796e071d794ab0ab0e490657346658 (127.10.97.66:42817)
I20260810 00:52:14.299746 21516 ts_manager.cc:194] Registered new tserver with Master: 81796e071d794ab0ab0e490657346658 (127.10.97.66:42817)
I20260810 00:52:14.300424 21236 ts_manager.cc:194] Registered new tserver with Master: 81796e071d794ab0ab0e490657346658 (127.10.97.66:42817)
I20260810 00:52:14.300791 20358 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:50535
I20260810 00:52:14.304414 22036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.263005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47196 (local address 127.10.97.67:38697)
0810 00:52:14.268286 (+ 5281us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:14.268293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:14.268313 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:52:14.268366 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:14.268372 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:14.268548 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:14.268745 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:14.270763 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.289747 (+ 18984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.291619 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.292459 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.292724 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.298450 (+ 5726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:14.298463 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:14.298479 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:14.298520 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:14.302023 (+ 3503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.302650 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.302659 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.302662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.302811 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.303103 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.303107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.303109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.303405 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:14.303623 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:14.303878 (+ 255us) server_negotiation.cc:299] Negotiation successful
0810 00:52:14.304076 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5148,"thread_start_us":4992,"threads_started":1}
I20260810 00:52:14.305426 21031 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:14.387612 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 20852
I20260810 00:52:14.409703 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:38697
--local_ip_for_outbound_sockets=127.10.97.67
--tserver_master_addrs=127.10.97.126:39955
--webserver_port=41117
--webserver_interface=127.10.97.67
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.67.keytab
--principal=oryx/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.97.126:39955,127.10.97.125:41889
--tserver_master_addrs=127.10.97.126:39955,127.10.97.125:41889,127.10.97.124:39429 with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:52:14.692818 21783 heartbeater.cc:503] Master 127.10.97.126:39955 was elected leader, sending a full tablet report...
W20260810 00:52:14.725610 22042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:14.725853 22042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:14.725950 22042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:14.732825 22042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:14.732923 22042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:14.733038 22042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:52:14.742615 22042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.67.keytab
--principal=oryx/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:38697
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=41117
--tserver_master_addrs=127.10.97.126:39955,127.10.97.125:41889,127.10.97.124:39429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:14.743738 22042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:14.745034 22042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:14.757373 22049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:14.757469 22056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:14.757339 22050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:14.758476 22042 server_base.cc:1034] running on GCE node
I20260810 00:52:14.759020 22042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:14.759855 22042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:14.761057 22042 hybrid_clock.cc:648] HybridClock initialized: now 1786323134761014 us; error 40 us; skew 500 ppm
I20260810 00:52:14.761349 22042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:14 dist-test-slave-jlzn krb5kdc[20291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323134, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:14.764225 22042 init.cc:377] Logged in from keytab as oryx/127.10.97.67@KRBTEST.COM (short username oryx)
I20260810 00:52:14.765535 22042 webserver.cc:460] Webserver started at http://127.10.97.67:41117/ using document root <none> and password file <none>
I20260810 00:52:14.766197 22042 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:14.766340 22042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:14.771291 22042 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260810 00:52:14.773761 22066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:14.774909 22042 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:52:14.775059 22042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "072edf83cafd41b68224a871315bb1c6"
format_stamp: "Formatted at 2026-08-10 00:52:11 on dist-test-slave-jlzn"
server_key: "6c27a8a7657ac313809f6eebd3d74956"
server_key_iv: "0540bba0f8633357134d9ec64022729a"
server_key_version: "encryptionkey@0"
I20260810 00:52:14.776007 22042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:14.788599 22042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:14.792186 22042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:14.792428 22042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:14.793277 22042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:14.794581 22042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:14.794675 22042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:14.794775 22042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:14.794847 22042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:14.824569 22042 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:38697
I20260810 00:52:14.824641 22235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:38697 every 8 connection(s)
I20260810 00:52:14.825981 22042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:52:14.831434 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 22042
Aug 10 00:52:14 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323134, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.124@KRBTEST.COM
Aug 10 00:52:14 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323134, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM
Aug 10 00:52:14 dist-test-slave-jlzn krb5kdc[20291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323134, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.125@KRBTEST.COM
I20260810 00:52:14.871896 22242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.828724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:56145 (local address 127.10.97.125:41889)
0810 00:52:14.848191 (+ 19467us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:14.848197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:14.848216 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:52:14.848274 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:14.848279 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:14.848364 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:14.848460 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:14.850771 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.851833 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.856471 (+ 4638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.857155 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.857253 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.862968 (+ 5715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:14.862974 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:14.862979 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:14.863018 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:14.866509 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.867979 (+ 1470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.867985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.867987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.868147 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.869103 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.869108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.869112 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.869479 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:14.869601 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:14.871254 (+ 1653us) server_negotiation.cc:299] Negotiation successful
0810 00:52:14.871470 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19352,"thread_start_us":19258,"threads_started":1}
I20260810 00:52:14.874182 22248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.829341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:41889 (local address 127.10.97.67:56145)
0810 00:52:14.839018 (+ 9677us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:14.839070 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:14.841532 (+ 2462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:14.849345 (+ 7813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:14.849350 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:14.850315 (+ 965us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:14.850591 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:14.850603 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.852019 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.852023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:14.855114 (+ 3091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:14.855137 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.858331 (+ 3194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.858338 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:14.858538 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.859266 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:14.859291 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:14.861987 (+ 2696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:14.866690 (+ 4703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:14.866700 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:14.866714 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:14.867460 (+ 746us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:14.868325 (+ 865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:14.868329 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:14.868332 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:14.868567 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:14.869740 (+ 1173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:14.869747 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:14.870111 (+ 364us) client_negotiation.cc:769] Sending connection context
0810 00:52:14.872362 (+ 2251us) client_negotiation.cc:240] Negotiation successful
0810 00:52:14.872867 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9397,"mutex_wait_us":159,"thread_start_us":17874,"threads_started":1}
I20260810 00:52:14.877731 22239 heartbeater.cc:347] Connected to a master server at 127.10.97.125:41889
I20260810 00:52:14.878038 22239 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:14.878902 22244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.828686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:46919 (local address 127.10.97.124:39429)
0810 00:52:14.845263 (+ 16577us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:14.845270 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:14.845292 (+ 22us) server_negotiation.cc:407] Connection header received
0810 00:52:14.845361 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:14.845367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:14.845462 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:14.845568 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:14.848388 (+ 2820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.849504 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.852017 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.852619 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.852755 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.861433 (+ 8678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:14.861440 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:14.861445 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:14.861489 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:14.875269 (+ 13780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.876179 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.876185 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.876189 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.876334 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.876782 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.876788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.876791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.877194 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:14.878162 (+ 968us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:14.878316 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 00:52:14.878535 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16442,"thread_start_us":18802,"threads_started":1}
I20260810 00:52:14.878904 22239 heartbeater.cc:511] Master 127.10.97.125:41889 requested a full tablet report, sending...
I20260810 00:52:14.879417 22249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.829882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39429 (local address 127.10.97.67:46919)
0810 00:52:14.842301 (+ 12419us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:14.842319 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:14.842433 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:14.845981 (+ 3548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:14.845992 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:14.847337 (+ 1345us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:14.848184 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:14.848211 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.849661 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.849670 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:14.851862 (+ 2192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:14.851875 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.853040 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.853048 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:14.853293 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.854023 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:14.854051 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:14.860453 (+ 6402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:14.875428 (+ 14975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:14.875434 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:14.875438 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:14.876001 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:14.876459 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:14.876463 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:14.876466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:14.876633 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:14.877356 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:14.877361 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:14.877522 (+ 161us) client_negotiation.cc:769] Sending connection context
0810 00:52:14.877665 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 00:52:14.877949 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12282,"thread_start_us":8768,"threads_started":1}
I20260810 00:52:14.880641 22236 heartbeater.cc:347] Connected to a master server at 127.10.97.124:39429
I20260810 00:52:14.880779 22236 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:14.881022 21236 ts_manager.cc:194] Registered new tserver with Master: 072edf83cafd41b68224a871315bb1c6 (127.10.97.67:38697)
I20260810 00:52:14.881230 22236 heartbeater.cc:511] Master 127.10.97.124:39429 requested a full tablet report, sending...
I20260810 00:52:14.882829 21516 ts_manager.cc:194] Registered new tserver with Master: 072edf83cafd41b68224a871315bb1c6 (127.10.97.67:38697)
I20260810 00:52:14.889123 22247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.830312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.67:52529)
0810 00:52:14.847156 (+ 16844us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:14.847172 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:14.847274 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:14.849222 (+ 1948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:14.849228 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:14.850019 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:14.850425 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:14.850438 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.851997 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.852003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:14.853818 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:14.853833 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.854814 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.854821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:14.855017 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.856068 (+ 1051us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:14.856089 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:14.858764 (+ 2675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:14.886249 (+ 27485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:14.886254 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:14.886257 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:14.886751 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:14.887291 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:14.887295 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:14.887299 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:14.887474 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:14.888145 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:14.888151 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:14.888300 (+ 149us) client_negotiation.cc:769] Sending connection context
0810 00:52:14.888441 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 00:52:14.888709 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16730,"mutex_wait_us":32,"thread_start_us":11680,"threads_started":1}
I20260810 00:52:14.890267 22241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.828680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:52529 (local address 127.10.97.126:39955)
0810 00:52:14.848891 (+ 20211us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:14.848898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:14.848915 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:52:14.848970 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:14.848975 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:14.849045 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:14.849149 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:14.850643 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.851831 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.853979 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.854601 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.854734 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.862045 (+ 7311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:14.862052 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:14.862057 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:14.862098 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:14.886089 (+ 23991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.886918 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.886923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.886926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.887161 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.887622 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.887626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.887630 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.888002 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:14.889504 (+ 1502us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:14.889648 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 00:52:14.889876 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20091,"thread_start_us":19991,"threads_started":1}
I20260810 00:52:14.891245 22237 heartbeater.cc:347] Connected to a master server at 127.10.97.126:39955
I20260810 00:52:14.891371 22237 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:14.891748 22237 heartbeater.cc:511] Master 127.10.97.126:39955 requested a full tablet report, sending...
I20260810 00:52:14.892611 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.856081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36852 (local address 127.10.97.124:39429)
0810 00:52:14.873532 (+ 17451us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:14.873539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:14.874077 (+ 538us) server_negotiation.cc:407] Connection header received
0810 00:52:14.874248 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:14.874254 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:14.874333 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:14.874426 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:14.877136 (+ 2710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.878293 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.880207 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.880819 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.880908 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.885770 (+ 4862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:14.885777 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:14.885782 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:14.885819 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:14.889103 (+ 3284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.889934 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.889939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.889942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.890096 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.890527 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.890533 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.890536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.890861 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:14.891080 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:14.892049 (+ 969us) server_negotiation.cc:299] Negotiation successful
0810 00:52:14.892277 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17324,"thread_start_us":17199,"threads_started":1}
I20260810 00:52:14.892807 20358 ts_manager.cc:194] Re-registered known tserver with Master: 072edf83cafd41b68224a871315bb1c6 (127.10.97.67:38697)
I20260810 00:52:14.893836 20358 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:52529
I20260810 00:52:14.899251 22242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.855661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54024 (local address 127.10.97.125:41889)
0810 00:52:14.872260 (+ 16599us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:14.872265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:14.884254 (+ 11989us) server_negotiation.cc:407] Connection header received
0810 00:52:14.884536 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:14.884541 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:14.884624 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:14.884709 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:14.886885 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.887987 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.889807 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.890441 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.890550 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.893985 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:14.893991 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:14.893996 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:14.894032 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:14.896866 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.897452 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.897457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.897460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.897613 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.897959 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.897964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.897968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.898282 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:14.898541 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:14.898717 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 00:52:14.898922 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16476,"thread_start_us":17891,"threads_started":1}
I20260810 00:52:14.901360 22260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.856047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37564 (local address 127.10.97.126:39955)
0810 00:52:14.882557 (+ 26510us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:14.882563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:14.882581 (+ 18us) server_negotiation.cc:407] Connection header received
0810 00:52:14.882639 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:14.882646 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:14.882724 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:14.882824 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:14.886291 (+ 3467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.887514 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.890432 (+ 2918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.891050 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.891145 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.895886 (+ 4741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:14.895892 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:14.895897 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:14.895933 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:14.898816 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.899579 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.899584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.899588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.899713 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:14.900091 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:14.900097 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:14.900100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:14.900363 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:14.900602 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:14.900843 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 00:52:14.901039 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26365,"thread_start_us":26693,"threads_started":1}
I20260810 00:52:14.904551 20358 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:52:14.911340 20358 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:52:14.937192 22284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.929024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47242 (local address 127.10.97.65:45981)
0810 00:52:14.929513 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:14.929521 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:14.929543 (+ 22us) server_negotiation.cc:407] Connection header received
0810 00:52:14.929624 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:14.929629 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:14.929803 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:14.929990 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:14.930882 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.932590 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.935034 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.936181 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.936304 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.936513 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:14.936748 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 00:52:14.936898 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":225,"threads_started":1}
I20260810 00:52:14.937340 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.928412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:45981 (local address 127.0.0.1:47242)
0810 00:52:14.929098 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:14.929116 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:14.929234 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:14.930233 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:14.930238 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:14.930263 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:14.930667 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:14.930681 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.932763 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.932769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:14.934891 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:14.934907 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.936395 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.936402 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:14.936566 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.936590 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 00:52:14.936679 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 00:52:14.936799 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":462,"threads_started":1}
I20260810 00:52:14.941216 21683 tablet_service.cc:1467] Processing CreateTablet for tablet f43cc2480dc0481d82dc18744d643e40 (DEFAULT_TABLE table=test-table [id=aa4c0c667ba542678db92879ecf3303c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:14.942756 21683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f43cc2480dc0481d82dc18744d643e40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:14.943351 22279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.927916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:38697 (local address 127.0.0.1:47200)
0810 00:52:14.928485 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:14.928505 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:14.928633 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:14.935893 (+ 7260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:14.935898 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:14.935927 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:14.936262 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:14.936272 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.938701 (+ 2429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.938710 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:14.940757 (+ 2047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:14.940770 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.942523 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.942529 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:14.942699 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.942733 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 00:52:14.942813 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 00:52:14.942937 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"spinlock_wait_cycles":384,"thread_start_us":408,"threads_started":1}
I20260810 00:52:14.943421 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.928508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47200 (local address 127.10.97.67:38697)
0810 00:52:14.932908 (+ 4400us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:14.932914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:14.932938 (+ 24us) server_negotiation.cc:407] Connection header received
0810 00:52:14.933012 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:14.933018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:14.933233 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:14.933367 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:14.937047 (+ 3680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.938570 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.940915 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.942320 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.942446 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.942652 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:14.942895 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 00:52:14.943053 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4271,"thread_start_us":4027,"threads_started":1}
I20260810 00:52:14.945866 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.930099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:42817 (local address 127.0.0.1:45422)
0810 00:52:14.937657 (+ 7558us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:14.937673 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:14.937787 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:14.938524 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:14.938528 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:14.938552 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:14.938946 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:14.938961 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.940827 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.940832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:14.942273 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:14.942286 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.945101 (+ 2815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.945108 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:14.945302 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.945333 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 00:52:14.945431 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 00:52:14.945550 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7453,"thread_start_us":7165,"threads_started":1}
I20260810 00:52:14.945904 22286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:14.930571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45422 (local address 127.10.97.66:42817)
0810 00:52:14.936745 (+ 6174us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:14.936755 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:14.937763 (+ 1008us) server_negotiation.cc:407] Connection header received
0810 00:52:14.937923 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:14.937930 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:14.938132 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:14.938326 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:14.939133 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.940664 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.943520 (+ 2856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:14.944867 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:14.945011 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:14.945210 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:14.945488 (+ 278us) server_negotiation.cc:299] Negotiation successful
0810 00:52:14.945638 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6040,"thread_start_us":5949,"threads_started":1}
I20260810 00:52:14.946825 22140 tablet_service.cc:1467] Processing CreateTablet for tablet f43cc2480dc0481d82dc18744d643e40 (DEFAULT_TABLE table=test-table [id=aa4c0c667ba542678db92879ecf3303c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:14.948161 22140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f43cc2480dc0481d82dc18744d643e40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:14.949523 21923 tablet_service.cc:1467] Processing CreateTablet for tablet f43cc2480dc0481d82dc18744d643e40 (DEFAULT_TABLE table=test-table [id=aa4c0c667ba542678db92879ecf3303c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:14.950759 21923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f43cc2480dc0481d82dc18744d643e40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:14.955302 22287 tablet_bootstrap.cc:492] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc: Bootstrap starting.
I20260810 00:52:14.958622 22287 tablet_bootstrap.cc:654] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:14.959614 22287 log.cc:824] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:14.962894 22287 tablet_bootstrap.cc:492] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc: No bootstrap required, opened a new log
I20260810 00:52:14.963385 22287 ts_tablet_manager.cc:1397] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260810 00:52:14.968250 22292 tablet_bootstrap.cc:492] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658: Bootstrap starting.
I20260810 00:52:14.968533 22290 tablet_bootstrap.cc:492] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6: Bootstrap starting.
I20260810 00:52:14.970961 22292 tablet_bootstrap.cc:654] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:14.971979 22292 log.cc:824] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:14.972033 22290 tablet_bootstrap.cc:654] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:14.972689 22287 raft_consensus.cc:348] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } } peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } }
I20260810 00:52:14.973121 22287 raft_consensus.cc:374] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:14.973172 22290 log.cc:824] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:14.973263 22287 raft_consensus.cc:729] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2cc23f25d1a4f7394bd756cf079b0fc, State: Initialized, Role: FOLLOWER
I20260810 00:52:14.973887 22287 consensus_queue.cc:260] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } } peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } }
I20260810 00:52:14.975517 22292 tablet_bootstrap.cc:492] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658: No bootstrap required, opened a new log
I20260810 00:52:14.975675 22287 ts_tablet_manager.cc:1428] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260810 00:52:14.975771 22292 ts_tablet_manager.cc:1397] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260810 00:52:14.984858 22292 raft_consensus.cc:348] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } } peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } }
I20260810 00:52:14.985281 22292 raft_consensus.cc:374] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:14.985426 22292 raft_consensus.cc:729] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81796e071d794ab0ab0e490657346658, State: Initialized, Role: FOLLOWER
I20260810 00:52:14.985455 22290 tablet_bootstrap.cc:492] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6: No bootstrap required, opened a new log
I20260810 00:52:14.985785 22290 ts_tablet_manager.cc:1397] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.012s
I20260810 00:52:14.986020 22292 consensus_queue.cc:260] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } } peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } }
I20260810 00:52:14.987608 22292 ts_tablet_manager.cc:1428] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
I20260810 00:52:14.987963 22022 heartbeater.cc:503] Master 127.10.97.126:39955 was elected leader, sending a full tablet report...
I20260810 00:52:14.994124 22290 raft_consensus.cc:348] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } } peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } }
I20260810 00:52:14.994429 22290 raft_consensus.cc:374] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:14.994520 22290 raft_consensus.cc:729] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 072edf83cafd41b68224a871315bb1c6, State: Initialized, Role: FOLLOWER
I20260810 00:52:14.994987 22290 consensus_queue.cc:260] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } } peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } }
I20260810 00:52:14.995970 22237 heartbeater.cc:503] Master 127.10.97.126:39955 was elected leader, sending a full tablet report...
I20260810 00:52:14.996452 22290 ts_tablet_manager.cc:1428] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6: Time spent starting tablet: real 0.011s user 0.009s sys 0.003s
W20260810 00:52:15.017518 22025 tablet.cc:2369] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:52:15.085201 22240 tablet.cc:2369] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:52:15.151224 21797 tablet.cc:2369] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:52:15.198421 22295 raft_consensus.cc:482] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:52:15.198721 22295 raft_consensus.cc:504] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } } peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } }
I20260810 00:52:15.200399 22295 leader_election.cc:288] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 072edf83cafd41b68224a871315bb1c6 (127.10.97.67:38697), 81796e071d794ab0ab0e490657346658 (127.10.97.66:42817)
I20260810 00:52:15.206984 22286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:15.200861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:50771 (local address 127.10.97.66:42817)
0810 00:52:15.201062 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:15.201068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:15.201741 (+ 673us) server_negotiation.cc:407] Connection header received
0810 00:52:15.201925 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:15.201929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:15.201988 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:15.202061 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:15.202657 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:15.203354 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:15.205322 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:15.206123 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:15.206273 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:15.206348 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:15.206616 (+ 268us) server_negotiation.cc:299] Negotiation successful
0810 00:52:15.206737 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 00:52:15.207062 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:15.200744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:47491 (local address 127.10.97.67:38697)
0810 00:52:15.200954 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:15.200962 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:15.201735 (+ 773us) server_negotiation.cc:407] Connection header received
0810 00:52:15.201972 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:15.201976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:15.202038 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:15.202113 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:15.202672 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:15.203371 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:15.205310 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:15.206202 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:15.206344 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:15.206415 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:15.206711 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 00:52:15.206833 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 00:52:15.207197 22303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:15.200831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:42817 (local address 127.10.97.65:50771)
0810 00:52:15.201587 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:15.201602 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:15.201790 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:15.202098 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:15.202102 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:15.202126 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:15.202472 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:15.202485 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:15.203484 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:15.203489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:15.205166 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:15.205178 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:15.206327 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:15.206333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:15.206471 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:15.206495 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 00:52:15.206572 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 00:52:15.206697 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"mutex_wait_us":85,"thread_start_us":462,"threads_started":1}
I20260810 00:52:15.207319 22302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:15.200574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:38697 (local address 127.10.97.65:47491)
0810 00:52:15.201568 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:15.201586 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:15.201840 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:15.202152 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:15.202156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:15.202184 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:15.202531 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:15.202540 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:15.203504 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:15.203510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:15.205158 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:15.205170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:15.206410 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:15.206417 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:15.206545 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:15.206567 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 00:52:15.206679 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 00:52:15.206795 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":824,"mutex_wait_us":16,"thread_start_us":621,"threads_started":1}
I20260810 00:52:15.208348 21960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f43cc2480dc0481d82dc18744d643e40" candidate_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81796e071d794ab0ab0e490657346658" is_pre_election: true
I20260810 00:52:15.208348 22168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f43cc2480dc0481d82dc18744d643e40" candidate_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "072edf83cafd41b68224a871315bb1c6" is_pre_election: true
I20260810 00:52:15.208895 21960 raft_consensus.cc:2454] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2cc23f25d1a4f7394bd756cf079b0fc in term 0.
I20260810 00:52:15.208895 22168 raft_consensus.cc:2454] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2cc23f25d1a4f7394bd756cf079b0fc in term 0.
I20260810 00:52:15.209583 21618 leader_election.cc:302] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 072edf83cafd41b68224a871315bb1c6, 81796e071d794ab0ab0e490657346658, c2cc23f25d1a4f7394bd756cf079b0fc; no voters:
I20260810 00:52:15.209955 22295 raft_consensus.cc:2781] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:52:15.210083 22295 raft_consensus.cc:482] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:52:15.210183 22295 raft_consensus.cc:3037] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:15.212489 22295 raft_consensus.cc:504] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } } peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } }
I20260810 00:52:15.213021 22295 leader_election.cc:288] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [CANDIDATE]: Term 1 election: Requested vote from peers 072edf83cafd41b68224a871315bb1c6 (127.10.97.67:38697), 81796e071d794ab0ab0e490657346658 (127.10.97.66:42817)
I20260810 00:52:15.213402 22168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f43cc2480dc0481d82dc18744d643e40" candidate_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "072edf83cafd41b68224a871315bb1c6"
I20260810 00:52:15.213459 21960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f43cc2480dc0481d82dc18744d643e40" candidate_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81796e071d794ab0ab0e490657346658"
I20260810 00:52:15.213570 22168 raft_consensus.cc:3037] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:15.213629 21960 raft_consensus.cc:3037] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:15.215914 21960 raft_consensus.cc:2454] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2cc23f25d1a4f7394bd756cf079b0fc in term 1.
I20260810 00:52:15.215975 22168 raft_consensus.cc:2454] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2cc23f25d1a4f7394bd756cf079b0fc in term 1.
I20260810 00:52:15.216321 21618 leader_election.cc:302] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81796e071d794ab0ab0e490657346658, c2cc23f25d1a4f7394bd756cf079b0fc; no voters:
I20260810 00:52:15.216591 22295 raft_consensus.cc:2781] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:52:15.216987 22295 raft_consensus.cc:686] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [term 1 LEADER]: Becoming Leader. State: Replica: c2cc23f25d1a4f7394bd756cf079b0fc, State: Running, Role: LEADER
I20260810 00:52:15.217447 22295 consensus_queue.cc:237] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } } peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } }
I20260810 00:52:15.221930 20358 catalog_manager.cc:5495] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc reported cstate change: term changed from 0 to 1, leader changed from <none> to c2cc23f25d1a4f7394bd756cf079b0fc (127.10.97.65). New cstate: current_term: 1 leader_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } health_report { overall_health: UNKNOWN } } }
I20260810 00:52:15.298611 22284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:15.293196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47252 (local address 127.10.97.65:45981)
0810 00:52:15.293403 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:15.293409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:15.293429 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:52:15.293537 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:15.293541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:15.293619 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:15.293788 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:52:15.294186 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:15.294896 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:15.296599 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:15.296997 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:15.297107 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:15.297587 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:52:15.298056 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:52:15.298200 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:15.298286 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 00:52:15.298368 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 00:52:15.306083 22168 raft_consensus.cc:1261] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6 [term 1 FOLLOWER]: Refusing update from remote peer c2cc23f25d1a4f7394bd756cf079b0fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:52:15.306108 21960 raft_consensus.cc:1261] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658 [term 1 FOLLOWER]: Refusing update from remote peer c2cc23f25d1a4f7394bd756cf079b0fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:52:15.306965 22304 consensus_queue.cc:1035] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:52:15.307299 22295 consensus_queue.cc:1035] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:52:15.318327 22312 mvcc.cc:205] Tried to move back new op lower bound from 7316779562205020160 to 7316779561853902848. Current Snapshot: MvccSnapshot[applied={T|T < 7316779562205020160}]
I20260810 00:52:15.335239 20357 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:52:15.335561 20357 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:52:15.343623 20357 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9c85df2445f54578af6ec069180550e6: Sending DeleteTablet for 3 replicas of tablet f43cc2480dc0481d82dc18744d643e40
I20260810 00:52:15.344818 22140 tablet_service.cc:1514] Processing DeleteTablet for tablet f43cc2480dc0481d82dc18744d643e40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:15 UTC) from {username='oryx', principal='oryx/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:47200
I20260810 00:52:15.345041 21683 tablet_service.cc:1514] Processing DeleteTablet for tablet f43cc2480dc0481d82dc18744d643e40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:15 UTC) from {username='oryx', principal='oryx/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:47242
I20260810 00:52:15.346161 22325 tablet_replica.cc:330] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc: stopping tablet replica
I20260810 00:52:15.346568 21923 tablet_service.cc:1514] Processing DeleteTablet for tablet f43cc2480dc0481d82dc18744d643e40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:15 UTC) from {username='oryx', principal='oryx/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:45422
I20260810 00:52:15.346673 22325 raft_consensus.cc:2229] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:52:15.347455 22325 raft_consensus.cc:2258] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:15.349506 22325 ts_tablet_manager.cc:1905] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:15.357901 22325 ts_tablet_manager.cc:1918] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:52:15.358218 22325 log.cc:1196] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/f43cc2480dc0481d82dc18744d643e40
I20260810 00:52:15.358927 22325 ts_tablet_manager.cc:1939] T f43cc2480dc0481d82dc18744d643e40 P c2cc23f25d1a4f7394bd756cf079b0fc: Deleting consensus metadata
I20260810 00:52:15.360280 22324 tablet_replica.cc:330] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6: stopping tablet replica
I20260810 00:52:15.360729 20339 catalog_manager.cc:4841] TS c2cc23f25d1a4f7394bd756cf079b0fc (127.10.97.65:45981): tablet f43cc2480dc0481d82dc18744d643e40 (table test-table [id=aa4c0c667ba542678db92879ecf3303c]) successfully deleted
I20260810 00:52:15.360806 22324 raft_consensus.cc:2229] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:52:15.361268 22324 raft_consensus.cc:2258] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:15.362812 22324 ts_tablet_manager.cc:1905] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:15.365134 22326 tablet_replica.cc:330] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658: stopping tablet replica
I20260810 00:52:15.365577 22326 raft_consensus.cc:2229] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:52:15.365990 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 20308
I20260810 00:52:15.366197 22326 raft_consensus.cc:2258] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:15.367009 20411 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:15.369186 22324 ts_tablet_manager.cc:1918] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:52:15.369412 22324 log.cc:1196] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/f43cc2480dc0481d82dc18744d643e40
I20260810 00:52:15.370002 22324 ts_tablet_manager.cc:1939] T f43cc2480dc0481d82dc18744d643e40 P 072edf83cafd41b68224a871315bb1c6: Deleting consensus metadata
I20260810 00:52:15.370561 22326 ts_tablet_manager.cc:1905] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:15.377056 22326 ts_tablet_manager.cc:1918] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:52:15.377209 22326 log.cc:1196] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/f43cc2480dc0481d82dc18744d643e40
I20260810 00:52:15.377743 22326 ts_tablet_manager.cc:1939] T f43cc2480dc0481d82dc18744d643e40 P 81796e071d794ab0ab0e490657346658: Deleting consensus metadata
I20260810 00:52:15.387202 22242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:15.352176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54032 (local address 127.10.97.125:41889)
0810 00:52:15.352345 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:15.352351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:15.364713 (+ 12362us) server_negotiation.cc:407] Connection header received
0810 00:52:15.364778 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:15.364784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:15.364862 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:15.364960 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:15.368898 (+ 3938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:15.370046 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:15.373621 (+ 3575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:15.374246 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:15.374377 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:15.380795 (+ 6418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:15.380802 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:15.380807 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:15.380849 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:15.384342 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:15.385119 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:15.385125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:15.385128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:15.385281 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:15.385674 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:15.385679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:15.385682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:15.386040 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:15.386293 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:15.386628 (+ 335us) server_negotiation.cc:299] Negotiation successful
0810 00:52:15.386850 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 00:52:15.402390 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:15.364468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36856 (local address 127.10.97.124:39429)
0810 00:52:15.364667 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:15.364674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:15.387060 (+ 22386us) server_negotiation.cc:407] Connection header received
0810 00:52:15.387292 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:15.387298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:15.387375 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:15.387439 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:15.389228 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:15.390324 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:15.392021 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:15.392658 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:15.392783 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:15.396178 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:15.396184 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:15.396188 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:15.396225 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:15.399552 (+ 3327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:15.400317 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:15.400322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:15.400327 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:15.400484 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:15.400916 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:15.400921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:15.400925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:15.401292 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:15.401530 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:15.401828 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 00:52:15.402066 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 00:52:15.497969 20335 catalog_manager.cc:4841] TS 81796e071d794ab0ab0e490657346658 (127.10.97.66:42817): tablet f43cc2480dc0481d82dc18744d643e40 (table test-table [id=aa4c0c667ba542678db92879ecf3303c]) successfully deleted
I20260810 00:52:15.498377 20333 catalog_manager.cc:4841] TS 072edf83cafd41b68224a871315bb1c6 (127.10.97.67:38697): tablet f43cc2480dc0481d82dc18744d643e40 (table test-table [id=aa4c0c667ba542678db92879ecf3303c]) successfully deleted
I20260810 00:52:15.499583 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 20308
I20260810 00:52:15.561048 22242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:15.530462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54040 (local address 127.10.97.125:41889)
0810 00:52:15.530671 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:15.530678 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:15.546085 (+ 15407us) server_negotiation.cc:407] Connection header received
0810 00:52:15.546710 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:15.546715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:15.546795 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:15.546894 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:15.549092 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:15.549967 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:15.551306 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:15.552016 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:15.552152 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:15.555070 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:15.555075 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:15.555078 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:15.555112 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:15.558252 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:15.559036 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:15.559042 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:15.559046 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:15.559186 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:15.559564 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:15.559569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:15.559573 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:15.559891 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:15.560151 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:15.560471 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 00:52:15.560702 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 00:52:15.561770 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:15.530983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36872 (local address 127.10.97.124:39429)
0810 00:52:15.531146 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:15.531151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:15.546217 (+ 15066us) server_negotiation.cc:407] Connection header received
0810 00:52:15.546533 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:15.546539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:15.546646 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:15.546724 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:15.548575 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:15.549444 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:15.551341 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:15.552016 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:15.552143 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:15.555570 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:15.555577 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:15.555581 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:15.555621 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:15.558836 (+ 3215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:15.559654 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:15.559661 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:15.559664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:15.559851 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:15.560287 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:15.560292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:15.560295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:15.560631 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:15.560858 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:15.561378 (+ 520us) server_negotiation.cc:299] Negotiation successful
0810 00:52:15.561532 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 00:52:16.326761 22338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.326034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.65:47599)
0810 00:52:16.326429 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.326559 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":200,"threads_started":1}
W20260810 00:52:16.327313 21783 heartbeater.cc:644] Failed to heartbeat to 127.10.97.126:39955 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
I20260810 00:52:16.328409 22339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.327644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.66:38597)
0810 00:52:16.328152 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.328258 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":232,"threads_started":1}
W20260810 00:52:16.328766 22022 heartbeater.cc:644] Failed to heartbeat to 127.10.97.126:39955 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
I20260810 00:52:16.331619 22340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.330897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.67:59863)
0810 00:52:16.331325 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.331449 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":205,"threads_started":1}
W20260810 00:52:16.331960 22237 heartbeater.cc:644] Failed to heartbeat to 127.10.97.126:39955 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
I20260810 00:52:16.842675 22341 raft_consensus.cc:482] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c85df2445f54578af6ec069180550e6)
I20260810 00:52:16.842959 22341 raft_consensus.cc:504] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: false } }
I20260810 00:52:16.844075 22343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c85df2445f54578af6ec069180550e6)
I20260810 00:52:16.844267 22343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: false } }
I20260810 00:52:16.844481 22341 leader_election.cc:288] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c85df2445f54578af6ec069180550e6 (127.10.97.126:39955), 997a5f1914d247a58a1261765413810e (127.10.97.125:41889)
I20260810 00:52:16.852823 22345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.844662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.0.0.1:38050)
0810 00:52:16.851350 (+ 6688us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.851601 (+ 251us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6535,"threads_started":1}
I20260810 00:52:16.853546 22343 leader_election.cc:288] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c85df2445f54578af6ec069180550e6 (127.10.97.126:39955), d04c2bdd7937490f9393f2c90c8fe33b (127.10.97.124:39429)
I20260810 00:52:16.853909 22349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.845286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.0.0.1:38064)
0810 00:52:16.853396 (+ 8110us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.853643 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7975,"threads_started":1}
W20260810 00:52:16.858731 21218 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
W20260810 00:52:16.861558 21493 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
I20260810 00:52:16.867599 22349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.851177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39429 (local address 127.0.0.1:36886)
0810 00:52:16.855114 (+ 3937us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.855139 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:16.855272 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:16.860396 (+ 5124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:16.860402 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:16.860436 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:16.860746 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:16.860758 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.862528 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.862535 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:16.864770 (+ 2235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:16.864788 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.866810 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.866817 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:16.867006 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:16.867044 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 00:52:16.867132 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 00:52:16.867273 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3831,"thread_start_us":1932,"threads_started":1}
I20260810 00:52:16.868657 22351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.852093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36886 (local address 127.10.97.124:39429)
0810 00:52:16.859928 (+ 7835us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:16.859935 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:16.859955 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:52:16.860023 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:16.860029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:16.860102 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:16.860199 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:16.860943 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.862348 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.864964 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.866388 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.866511 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:16.866625 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:16.868160 (+ 1535us) server_negotiation.cc:299] Negotiation successful
0810 00:52:16.868313 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7627,"thread_start_us":7725,"threads_started":1}
I20260810 00:52:16.869371 21536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "997a5f1914d247a58a1261765413810e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" is_pre_election: true
I20260810 00:52:16.869758 21536 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 997a5f1914d247a58a1261765413810e in term 1.
I20260810 00:52:16.870414 21217 leader_election.cc:302] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 997a5f1914d247a58a1261765413810e, d04c2bdd7937490f9393f2c90c8fe33b; no voters:
I20260810 00:52:16.871052 22343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 00:52:16.871264 22343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c85df2445f54578af6ec069180550e6)
I20260810 00:52:16.871439 22343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:52:16.871950 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.871348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.0.0.1:38092)
0810 00:52:16.871535 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.871742 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 00:52:16.872263 21493 leader_election.cc:334] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c85df2445f54578af6ec069180550e6 (127.10.97.126:39955): Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
I20260810 00:52:16.872843 22349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.872309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.0.0.1:38072)
0810 00:52:16.872528 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.872627 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260810 00:52:16.873087 21218 leader_election.cc:334] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c85df2445f54578af6ec069180550e6 (127.10.97.126:39955): Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
I20260810 00:52:16.874980 22343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: false } }
I20260810 00:52:16.876009 21536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "997a5f1914d247a58a1261765413810e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d04c2bdd7937490f9393f2c90c8fe33b"
I20260810 00:52:16.876176 22349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.875692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.0.0.1:38094)
0810 00:52:16.875904 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.876024 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260810 00:52:16.876235 21536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:52:16.876583 22343 leader_election.cc:288] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [CANDIDATE]: Term 2 election: Requested vote from peers 9c85df2445f54578af6ec069180550e6 (127.10.97.126:39955), d04c2bdd7937490f9393f2c90c8fe33b (127.10.97.124:39429)
I20260810 00:52:16.877509 22349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.877067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.0.0.1:38110)
0810 00:52:16.877265 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.877351 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 00:52:16.877712 21218 leader_election.cc:334] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c85df2445f54578af6ec069180550e6 (127.10.97.126:39955): Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
I20260810 00:52:16.878635 21536 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997a5f1914d247a58a1261765413810e in term 2.
I20260810 00:52:16.879055 22345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.844858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:41889 (local address 127.0.0.1:54050)
0810 00:52:16.853293 (+ 8435us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.853313 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:16.853457 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:16.873290 (+ 19833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:16.873296 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:16.873323 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:16.873521 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:16.873533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.875020 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.875023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:16.876924 (+ 1901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:16.876940 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.878388 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.878395 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:16.878544 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:16.878568 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 00:52:16.878652 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 00:52:16.878788 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8339,"thread_start_us":6961,"threads_started":1}
I20260810 00:52:16.879042 21217 leader_election.cc:302] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 997a5f1914d247a58a1261765413810e, d04c2bdd7937490f9393f2c90c8fe33b; no voters: 9c85df2445f54578af6ec069180550e6
I20260810 00:52:16.879242 22347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.844853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54050 (local address 127.10.97.125:41889)
0810 00:52:16.872938 (+ 28085us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:16.872944 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:16.872963 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:52:16.873022 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:16.873027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:16.873112 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:16.873248 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:16.873682 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.874897 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.877112 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.878201 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.878321 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:16.878423 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:16.878693 (+ 270us) server_negotiation.cc:299] Negotiation successful
0810 00:52:16.878875 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27958,"thread_start_us":27773,"threads_started":1}
I20260810 00:52:16.879405 22343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 2 FOLLOWER]: Leader election won for term 2
I20260810 00:52:16.879863 22343 raft_consensus.cc:686] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [term 2 LEADER]: Becoming Leader. State: Replica: 997a5f1914d247a58a1261765413810e, State: Running, Role: LEADER
I20260810 00:52:16.880052 21250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "997a5f1914d247a58a1261765413810e" is_pre_election: true
I20260810 00:52:16.880308 22343 consensus_queue.cc:237] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: false } }
I20260810 00:52:16.880770 21495 leader_election.cc:302] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d04c2bdd7937490f9393f2c90c8fe33b; no voters: 997a5f1914d247a58a1261765413810e, 9c85df2445f54578af6ec069180550e6
I20260810 00:52:16.881098 22341 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 00:52:16.882246 22355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 997a5f1914d247a58a1261765413810e. Latest consensus state: current_term: 2 leader_uuid: "997a5f1914d247a58a1261765413810e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: false } } }
I20260810 00:52:16.882642 22355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:16.883271 22358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:52:16.886783 22358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:52:16.887554 22358 catalog_manager.cc:1259] Loaded cluster ID: 94d15d9e56fe40589d69ace7b6a3507e
I20260810 00:52:16.887625 22358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:52:16.888760 22358 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:52:16.889523 22358 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: Loaded TSK: 0
I20260810 00:52:16.890290 22358 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:52:16.932917 21236 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54040:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:52:16.939941 21236 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:52:16.943687 22349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.943341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.0.0.1:38124)
0810 00:52:16.943485 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.943557 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 00:52:16.943886 21536 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [term 2 FOLLOWER]: Refusing update from remote peer 997a5f1914d247a58a1261765413810e: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260810 00:52:16.944536 22355 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260810 00:52:16.944746 22349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.944318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.0.0.1:38134)
0810 00:52:16.944455 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.944566 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 00:52:16.944905 21218 consensus_peers.cc:487] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e -> Peer 9c85df2445f54578af6ec069180550e6 (127.10.97.126:39955): Couldn't send request to peer 9c85df2445f54578af6ec069180550e6. Status: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 00:52:16.947191 22341 sys_catalog.cc:455] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 997a5f1914d247a58a1261765413810e. Latest consensus state: current_term: 2 leader_uuid: "997a5f1914d247a58a1261765413810e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: false } } }
I20260810 00:52:16.947415 22341 sys_catalog.cc:458] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:52:16.948673 22355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "997a5f1914d247a58a1261765413810e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: false } } }
I20260810 00:52:16.948935 22355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:16.952189 22361 mvcc.cc:205] Tried to move back new op lower bound from 7316779568915881984 to 7316779568665649152. Current Snapshot: MvccSnapshot[applied={T|T < 7316779568915881984}]
I20260810 00:52:16.956530 22341 sys_catalog.cc:455] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "997a5f1914d247a58a1261765413810e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9c85df2445f54578af6ec069180550e6" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39955 } } peers { permanent_uuid: "997a5f1914d247a58a1261765413810e" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 41889 } attrs { promote: false } } peers { permanent_uuid: "d04c2bdd7937490f9393f2c90c8fe33b" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39429 } attrs { promote: false } } }
I20260810 00:52:16.956727 22341 sys_catalog.cc:458] T 00000000000000000000000000000000 P d04c2bdd7937490f9393f2c90c8fe33b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:52:16.968128 22349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.960934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:45981 (local address 127.0.0.1:47268)
0810 00:52:16.961089 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.961106 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:16.961222 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:16.962209 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:16.962215 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:16.962249 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:16.962446 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:16.962458 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.964081 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.964087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:16.966056 (+ 1969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:16.966068 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.967415 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.967421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:16.967597 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:16.967622 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 00:52:16.967707 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 00:52:16.967846 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 00:52:16.968811 22366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.961001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47268 (local address 127.10.97.65:45981)
0810 00:52:16.961874 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:16.961880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:16.961898 (+ 18us) server_negotiation.cc:407] Connection header received
0810 00:52:16.961965 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:16.961970 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:16.962044 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:16.962597 (+ 553us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:16.962797 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.963927 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.966206 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.967274 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.967385 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:16.967497 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:16.968416 (+ 919us) server_negotiation.cc:299] Negotiation successful
0810 00:52:16.968554 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":640,"threads_started":1}
I20260810 00:52:16.969576 21683 tablet_service.cc:1467] Processing CreateTablet for tablet 3ed7593bc8244da7822112261e5289ec (DEFAULT_TABLE table=test-table [id=c262328ab4054b2281385d514bd056a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:16.970034 21683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ed7593bc8244da7822112261e5289ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:16.972293 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.962110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45428 (local address 127.10.97.66:42817)
0810 00:52:16.965932 (+ 3822us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:16.965956 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:16.965977 (+ 21us) server_negotiation.cc:407] Connection header received
0810 00:52:16.966051 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:16.966057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:16.966130 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:16.966233 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:16.966804 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.967848 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.970092 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.971254 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.971345 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:16.971452 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:16.971871 (+ 419us) server_negotiation.cc:299] Negotiation successful
0810 00:52:16.972003 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3671,"thread_start_us":3564,"threads_started":1}
I20260810 00:52:16.972379 22368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.961722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:42817 (local address 127.0.0.1:45428)
0810 00:52:16.962250 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.962266 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:16.962381 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:16.966242 (+ 3861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:16.966246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:16.966273 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:16.966625 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:16.966636 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.967998 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.968007 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:16.969905 (+ 1898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:16.969919 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.971548 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.971554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:16.971699 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:16.971722 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 00:52:16.971818 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 00:52:16.971981 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":3,"thread_start_us":479,"threads_started":1}
I20260810 00:52:16.973402 21923 tablet_service.cc:1467] Processing CreateTablet for tablet 3ed7593bc8244da7822112261e5289ec (DEFAULT_TABLE table=test-table [id=c262328ab4054b2281385d514bd056a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:16.973795 21923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ed7593bc8244da7822112261e5289ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:16.975728 22348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.961303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:38697 (local address 127.0.0.1:47216)
0810 00:52:16.961639 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.961655 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:16.961771 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:16.966990 (+ 5219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:16.966995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:16.967018 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:16.967339 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:16.967352 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.971349 (+ 3997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.971354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:16.973447 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:16.973461 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.975012 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.975018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:16.975182 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:16.975216 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 00:52:16.975305 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 00:52:16.975427 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":21}
I20260810 00:52:16.975778 22369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.961992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47216 (local address 127.10.97.67:38697)
0810 00:52:16.966667 (+ 4675us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:16.966671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:16.966686 (+ 15us) server_negotiation.cc:407] Connection header received
0810 00:52:16.966758 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:16.966762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:16.966833 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:16.967869 (+ 1036us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:16.968150 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.971201 (+ 3051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.973646 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:16.974794 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:16.974915 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:16.975004 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:16.975347 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 00:52:16.975490 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4567,"thread_start_us":4477,"threads_started":1}
I20260810 00:52:16.976604 22371 tablet_bootstrap.cc:492] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc: Bootstrap starting.
I20260810 00:52:16.977867 22371 tablet_bootstrap.cc:654] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:16.978981 22373 tablet_bootstrap.cc:492] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658: Bootstrap starting.
I20260810 00:52:16.979349 22140 tablet_service.cc:1467] Processing CreateTablet for tablet 3ed7593bc8244da7822112261e5289ec (DEFAULT_TABLE table=test-table [id=c262328ab4054b2281385d514bd056a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:16.979677 22371 tablet_bootstrap.cc:492] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc: No bootstrap required, opened a new log
I20260810 00:52:16.979766 22140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ed7593bc8244da7822112261e5289ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:16.979836 22371 ts_tablet_manager.cc:1397] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 00:52:16.980736 22371 raft_consensus.cc:348] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } } peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } }
I20260810 00:52:16.980901 22371 raft_consensus.cc:374] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:16.980955 22371 raft_consensus.cc:729] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2cc23f25d1a4f7394bd756cf079b0fc, State: Initialized, Role: FOLLOWER
I20260810 00:52:16.980993 22373 tablet_bootstrap.cc:654] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:16.981128 22371 consensus_queue.cc:260] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } } peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } }
I20260810 00:52:16.981843 22371 ts_tablet_manager.cc:1428] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 00:52:16.983196 22377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.982288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.65:35347)
0810 00:52:16.982767 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.982959 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"threads_started":1}
I20260810 00:52:16.986488 21793 heartbeater.cc:503] Master 127.10.97.125:41889 was elected leader, sending a full tablet report...
I20260810 00:52:16.988060 22377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.986637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.65:37353)
0810 00:52:16.987849 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.987930 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1072}
I20260810 00:52:16.988819 22379 tablet_bootstrap.cc:492] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6: Bootstrap starting.
I20260810 00:52:16.989850 22373 tablet_bootstrap.cc:492] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658: No bootstrap required, opened a new log
I20260810 00:52:16.989979 22373 ts_tablet_manager.cc:1397] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260810 00:52:16.990176 22379 tablet_bootstrap.cc:654] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:16.990589 22373 raft_consensus.cc:348] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } } peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } }
I20260810 00:52:16.990789 22373 raft_consensus.cc:374] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:16.990926 22373 raft_consensus.cc:729] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81796e071d794ab0ab0e490657346658, State: Initialized, Role: FOLLOWER
I20260810 00:52:16.991183 22373 consensus_queue.cc:260] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } } peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } }
I20260810 00:52:16.991981 22379 tablet_bootstrap.cc:492] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6: No bootstrap required, opened a new log
I20260810 00:52:16.992031 22373 ts_tablet_manager.cc:1428] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260810 00:52:16.992185 22379 ts_tablet_manager.cc:1397] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260810 00:52:16.993000 22379 raft_consensus.cc:348] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } } peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } }
I20260810 00:52:16.993288 22379 raft_consensus.cc:374] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:16.993440 22379 raft_consensus.cc:729] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 072edf83cafd41b68224a871315bb1c6, State: Initialized, Role: FOLLOWER
I20260810 00:52:16.993789 22379 consensus_queue.cc:260] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } } peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } }
I20260810 00:52:16.994350 22023 heartbeater.cc:503] Master 127.10.97.125:41889 was elected leader, sending a full tablet report...
I20260810 00:52:16.995959 22383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.992235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.66:51567)
0810 00:52:16.995542 (+ 3307us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.995715 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3186,"thread_start_us":3072,"threads_started":1}
I20260810 00:52:16.996670 22379 ts_tablet_manager.cc:1428] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260810 00:52:16.997640 22383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.996662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.66:52731)
0810 00:52:16.997422 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.997506 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":660}
I20260810 00:52:16.997830 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.996921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.67:53941)
0810 00:52:16.997454 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.997576 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":271,"threads_started":1}
I20260810 00:52:16.998662 22239 heartbeater.cc:503] Master 127.10.97.125:41889 was elected leader, sending a full tablet report...
I20260810 00:52:16.998858 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:16.998467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.67:50067)
0810 00:52:16.998658 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:16.998738 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 00:52:17.022441 22381 raft_consensus.cc:482] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:52:17.022600 22381 raft_consensus.cc:504] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } } peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } }
I20260810 00:52:17.024111 22381 leader_election.cc:288] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2cc23f25d1a4f7394bd756cf079b0fc (127.10.97.65:45981), 072edf83cafd41b68224a871315bb1c6 (127.10.97.67:38697)
I20260810 00:52:17.030630 22366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:17.024362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:51053 (local address 127.10.97.65:45981)
0810 00:52:17.024536 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:17.024542 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:17.024561 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:52:17.024721 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:17.024727 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:17.024802 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:17.024895 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:17.025395 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:17.026326 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:17.028764 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:17.029895 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:17.030056 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:17.030126 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:17.030359 (+ 233us) server_negotiation.cc:299] Negotiation successful
0810 00:52:17.030449 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 00:52:17.030704 22383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:17.024264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:45981 (local address 127.10.97.66:51053)
0810 00:52:17.024426 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:17.024445 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:17.024577 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:17.024929 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:17.024933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:17.024958 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:17.025271 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:17.025282 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:17.026448 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:17.026452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:17.028620 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:17.028635 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:17.030047 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:17.030051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:17.030182 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:17.030215 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 00:52:17.030312 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 00:52:17.030446 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":41}
I20260810 00:52:17.030992 22369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:17.024365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:41843 (local address 127.10.97.67:38697)
0810 00:52:17.024567 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:17.024572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:17.025107 (+ 535us) server_negotiation.cc:407] Connection header received
0810 00:52:17.025246 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:17.025249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:17.025297 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:17.025377 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:17.025788 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:17.026676 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:17.028964 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:17.030101 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:17.030230 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:17.030277 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:17.030670 (+ 393us) server_negotiation.cc:299] Negotiation successful
0810 00:52:17.030793 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 00:52:17.031086 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:17.024525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:38697 (local address 127.10.97.66:41843)
0810 00:52:17.025000 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:17.025016 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:17.025143 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:17.025400 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:17.025403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:17.025423 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:17.025664 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:17.025671 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:17.026783 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:17.026788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:17.028841 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:17.028851 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:17.030315 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:17.030322 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:17.030498 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:17.030531 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 00:52:17.030611 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 00:52:17.030736 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":34,"thread_start_us":222,"threads_started":1}
I20260810 00:52:17.031724 22168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ed7593bc8244da7822112261e5289ec" candidate_uuid: "81796e071d794ab0ab0e490657346658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "072edf83cafd41b68224a871315bb1c6" is_pre_election: true
I20260810 00:52:17.031708 21715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ed7593bc8244da7822112261e5289ec" candidate_uuid: "81796e071d794ab0ab0e490657346658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" is_pre_election: true
I20260810 00:52:17.031993 22168 raft_consensus.cc:2454] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81796e071d794ab0ab0e490657346658 in term 0.
I20260810 00:52:17.032130 21715 raft_consensus.cc:2454] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81796e071d794ab0ab0e490657346658 in term 0.
I20260810 00:52:17.032493 21828 leader_election.cc:302] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 072edf83cafd41b68224a871315bb1c6, 81796e071d794ab0ab0e490657346658; no voters:
I20260810 00:52:17.032840 22381 raft_consensus.cc:2781] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:52:17.032948 22381 raft_consensus.cc:482] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:52:17.033036 22381 raft_consensus.cc:3037] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:17.035215 22381 raft_consensus.cc:504] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } } peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } }
I20260810 00:52:17.035777 22381 leader_election.cc:288] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [CANDIDATE]: Term 1 election: Requested vote from peers c2cc23f25d1a4f7394bd756cf079b0fc (127.10.97.65:45981), 072edf83cafd41b68224a871315bb1c6 (127.10.97.67:38697)
I20260810 00:52:17.036121 21715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ed7593bc8244da7822112261e5289ec" candidate_uuid: "81796e071d794ab0ab0e490657346658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc"
I20260810 00:52:17.036173 22168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ed7593bc8244da7822112261e5289ec" candidate_uuid: "81796e071d794ab0ab0e490657346658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "072edf83cafd41b68224a871315bb1c6"
I20260810 00:52:17.036309 21715 raft_consensus.cc:3037] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:17.036350 22168 raft_consensus.cc:3037] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:17.039096 21715 raft_consensus.cc:2454] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81796e071d794ab0ab0e490657346658 in term 1.
I20260810 00:52:17.039096 22168 raft_consensus.cc:2454] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81796e071d794ab0ab0e490657346658 in term 1.
I20260810 00:52:17.039474 21828 leader_election.cc:302] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 072edf83cafd41b68224a871315bb1c6, 81796e071d794ab0ab0e490657346658; no voters:
I20260810 00:52:17.039736 22381 raft_consensus.cc:2781] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:52:17.040148 22381 raft_consensus.cc:686] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [term 1 LEADER]: Becoming Leader. State: Replica: 81796e071d794ab0ab0e490657346658, State: Running, Role: LEADER
I20260810 00:52:17.040503 22381 consensus_queue.cc:237] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } } peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } }
I20260810 00:52:17.040952 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:17.040614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.66:44275)
0810 00:52:17.040749 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:17.040849 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 00:52:17.043452 21236 catalog_manager.cc:5495] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81796e071d794ab0ab0e490657346658 (127.10.97.66). New cstate: current_term: 1 leader_uuid: "81796e071d794ab0ab0e490657346658" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81796e071d794ab0ab0e490657346658" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42817 } health_report { overall_health: HEALTHY } } }
I20260810 00:52:17.062951 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:17.058795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45436 (local address 127.10.97.66:42817)
0810 00:52:17.058944 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:17.058948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:17.058964 (+ 16us) server_negotiation.cc:407] Connection header received
0810 00:52:17.059052 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:17.059056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:17.059120 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:17.059233 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:52:17.059480 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:17.060172 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:17.061328 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:17.061712 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:17.061797 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:17.062100 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:52:17.062568 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:52:17.062676 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:17.062714 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 00:52:17.062770 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 00:52:17.066498 22168 raft_consensus.cc:1261] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6 [term 1 FOLLOWER]: Refusing update from remote peer 81796e071d794ab0ab0e490657346658: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:52:17.066922 21715 raft_consensus.cc:1261] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc [term 1 FOLLOWER]: Refusing update from remote peer 81796e071d794ab0ab0e490657346658: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:52:17.067111 22392 consensus_queue.cc:1035] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [LEADER]: Connected to new peer: Peer: permanent_uuid: "072edf83cafd41b68224a871315bb1c6" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:52:17.067638 22381 consensus_queue.cc:1035] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2cc23f25d1a4f7394bd756cf079b0fc" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 45981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:52:17.070209 22377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:17.069680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.65:57091)
0810 00:52:17.069911 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:17.070002 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260810 00:52:17.070284 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:17.069773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.67:55309)
0810 00:52:17.069981 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:17.070092 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260810 00:52:17.075835 22399 mvcc.cc:205] Tried to move back new op lower bound from 7316779569418510336 to 7316779569321107456. Current Snapshot: MvccSnapshot[applied={T|T < 7316779569418510336}]
I20260810 00:52:17.076699 22377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:17.076254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.65:43349)
0810 00:52:17.076462 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:17.076543 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260810 00:52:17.077634 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:17.077176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.67:38467)
0810 00:52:17.077385 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:17.077463 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260810 00:52:17.079164 22398 mvcc.cc:205] Tried to move back new op lower bound from 7316779569418510336 to 7316779569321107456. Current Snapshot: MvccSnapshot[applied={T|T < 7316779569418510336}]
I20260810 00:52:17.084307 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:17.083758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.66:36965)
0810 00:52:17.084056 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:17.084162 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204}
I20260810 00:52:17.086773 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:17.085010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.10.97.66:41655)
0810 00:52:17.086526 (+ 1516us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:17.086633 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1423}
I20260810 00:52:17.093137 21236 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:52:17.093362 21236 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:52:17.098459 21236 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 997a5f1914d247a58a1261765413810e: Sending DeleteTablet for 3 replicas of tablet 3ed7593bc8244da7822112261e5289ec
I20260810 00:52:17.099519 21683 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ed7593bc8244da7822112261e5289ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:17 UTC) from {username='oryx', principal='oryx/127.10.97.125@KRBTEST.COM'} at 127.0.0.1:47268
I20260810 00:52:17.099699 22140 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ed7593bc8244da7822112261e5289ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:17 UTC) from {username='oryx', principal='oryx/127.10.97.125@KRBTEST.COM'} at 127.0.0.1:47216
I20260810 00:52:17.100119 21923 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ed7593bc8244da7822112261e5289ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:17 UTC) from {username='oryx', principal='oryx/127.10.97.125@KRBTEST.COM'} at 127.0.0.1:45428
I20260810 00:52:17.100203 22411 tablet_replica.cc:330] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc: stopping tablet replica
I20260810 00:52:17.100363 22411 raft_consensus.cc:2229] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:52:17.100701 22411 raft_consensus.cc:2258] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:17.100791 22413 tablet_replica.cc:330] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658: stopping tablet replica
I20260810 00:52:17.101032 22413 raft_consensus.cc:2229] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:52:17.101662 22413 raft_consensus.cc:2258] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:17.101953 22411 ts_tablet_manager.cc:1905] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:17.102689 22413 ts_tablet_manager.cc:1905] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:17.104023 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 21474
I20260810 00:52:17.109417 22413 ts_tablet_manager.cc:1918] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:52:17.109622 22413 log.cc:1196] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/3ed7593bc8244da7822112261e5289ec
I20260810 00:52:17.110180 22413 ts_tablet_manager.cc:1939] T 3ed7593bc8244da7822112261e5289ec P 81796e071d794ab0ab0e490657346658: Deleting consensus metadata
I20260810 00:52:17.111364 21219 catalog_manager.cc:4841] TS 81796e071d794ab0ab0e490657346658 (127.10.97.66:42817): tablet 3ed7593bc8244da7822112261e5289ec (table test-table [id=c262328ab4054b2281385d514bd056a1]) successfully deleted
I20260810 00:52:17.114758 22411 ts_tablet_manager.cc:1918] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:52:17.114890 22411 log.cc:1196] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/3ed7593bc8244da7822112261e5289ec
I20260810 00:52:17.115028 22412 tablet_replica.cc:330] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6: stopping tablet replica
I20260810 00:52:17.115294 22412 raft_consensus.cc:2229] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:52:17.115438 22411 ts_tablet_manager.cc:1939] T 3ed7593bc8244da7822112261e5289ec P c2cc23f25d1a4f7394bd756cf079b0fc: Deleting consensus metadata
I20260810 00:52:17.115646 22412 raft_consensus.cc:2258] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:17.116737 21221 catalog_manager.cc:4841] TS c2cc23f25d1a4f7394bd756cf079b0fc (127.10.97.65:45981): tablet 3ed7593bc8244da7822112261e5289ec (table test-table [id=c262328ab4054b2281385d514bd056a1]) successfully deleted
I20260810 00:52:17.117125 22412 ts_tablet_manager.cc:1905] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:17.121742 22412 ts_tablet_manager.cc:1918] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:52:17.121870 22412 log.cc:1196] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/3ed7593bc8244da7822112261e5289ec
I20260810 00:52:17.122300 22412 ts_tablet_manager.cc:1939] T 3ed7593bc8244da7822112261e5289ec P 072edf83cafd41b68224a871315bb1c6: Deleting consensus metadata
I20260810 00:52:17.123337 21218 catalog_manager.cc:4841] TS 072edf83cafd41b68224a871315bb1c6 (127.10.97.67:38697): tablet 3ed7593bc8244da7822112261e5289ec (table test-table [id=c262328ab4054b2281385d514bd056a1]) successfully deleted
I20260810 00:52:17.131278 22366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:17.105603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47280 (local address 127.10.97.65:45981)
0810 00:52:17.105783 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:17.105790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:17.116487 (+ 10697us) server_negotiation.cc:407] Connection header received
0810 00:52:17.116730 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:17.116736 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:17.116823 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:17.116915 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:17.118774 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:17.119886 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:17.121737 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:17.122325 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:17.122452 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:17.125864 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:17.125877 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:17.125892 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:17.125927 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:17.128761 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:17.129511 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:17.129518 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:17.129520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:17.129640 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:17.129990 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:17.129993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:17.129995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:17.130344 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:17.130562 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:17.130875 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 00:52:17.131030 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 00:52:17.132061 21771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:17.250191 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 21474
I20260810 00:52:17.286083 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 21799
I20260810 00:52:17.299929 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:17.286801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45444 (local address 127.10.97.66:42817)
0810 00:52:17.286981 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:17.286987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:17.287143 (+ 156us) server_negotiation.cc:407] Connection header received
0810 00:52:17.287326 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:17.287330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:17.287408 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:17.287511 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:17.288878 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:17.289679 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:17.290909 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:17.291340 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:17.291448 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:17.294048 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:17.294064 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:17.294082 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:17.294132 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:17.297129 (+ 2997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:17.297785 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:17.297796 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:17.297798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:17.297986 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:17.298366 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:17.298369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:17.298371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:17.298752 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:17.299073 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:17.299355 (+ 282us) server_negotiation.cc:299] Negotiation successful
0810 00:52:17.299568 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 00:52:17.300655 22015 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:17.345911 22348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:17.345409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.0.0.1:38140)
0810 00:52:17.345615 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:17.345725 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 00:52:17.346918 22348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:17.346522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39955 (local address 127.0.0.1:38154)
0810 00:52:17.346661 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:17.346763 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:39955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 00:52:17.416030 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 21799
I20260810 00:52:17.441681 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 22042
I20260810 00:52:17.453963 22369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:17.442367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47226 (local address 127.10.97.67:38697)
0810 00:52:17.442519 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:17.442523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:17.442541 (+ 18us) server_negotiation.cc:407] Connection header received
0810 00:52:17.442687 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:17.442691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:17.442769 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:17.442853 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:17.444147 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:17.444885 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:17.446019 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:17.446421 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:17.446507 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:17.449022 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:17.449037 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:17.449055 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:17.449085 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:17.451702 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:17.452364 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:17.452371 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:17.452373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:17.452494 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:17.452781 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:17.452783 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:17.452785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:17.453113 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:17.453346 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:17.453544 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 00:52:17.453714 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 00:52:17.454602 22228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:17.545588 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 22042
I20260810 00:52:17.563225 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 21046
I20260810 00:52:17.564289 21277 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:17.683393 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 21046
W20260810 00:52:17.711613 21495 connection.cc:535] server connection from 127.0.0.1:36886 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 00:52:17.712049 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 21190
I20260810 00:52:17.713068 21563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:17.827977 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 21190
2026-08-10T00:52:17Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9380 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:52:17 dist-test-slave-jlzn krb5kdc[22426](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:52:17 dist-test-slave-jlzn krb5kdc[22426](info): set up 2 sockets
Aug 10 00:52:17 dist-test-slave-jlzn krb5kdc[22426](info): commencing operation
krb5kdc: starting...
W20260810 00:52:19.010280 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:52:19 dist-test-slave-jlzn krb5kdc[22426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:52:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:52:19Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.126.keytab.
Entry for principal oryx/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.126.keytab.
I20260810 00:52:19.224294 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:42341
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.126.keytab
--principal=oryx/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:34233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:42341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:19.529429 22446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:19.529691 22446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:19.529815 22446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:19.536654 22446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:52:19.536746 22446 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 00:52:19.536810 22446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:19.536893 22446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:19.536957 22446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:52:19.537039 22446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:52:19.546237 22446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:34233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:42341
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.126.keytab
--principal=oryx/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:42341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:19.547266 22446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:19.548489 22446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:19.559186 22452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:19.559214 22457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:19.559248 22453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:19.560488 22446 server_base.cc:1034] running on GCE node
I20260810 00:52:19.561190 22446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:19.562371 22446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:19.563572 22446 hybrid_clock.cc:648] HybridClock initialized: now 1786323139563503 us; error 44 us; skew 500 ppm
I20260810 00:52:19.563912 22446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:19 dist-test-slave-jlzn krb5kdc[22426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323139, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:19.566848 22446 init.cc:377] Logged in from keytab as oryx/127.10.97.126@KRBTEST.COM (short username oryx)
I20260810 00:52:19.568177 22446 webserver.cc:460] Webserver started at http://127.10.97.126:34343/ using document root <none> and password file <none>
I20260810 00:52:19.568809 22446 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:19.568908 22446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:19.569211 22446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:19.572001 22446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "cbde9124c7bb4285994cfc3e1314a80b"
format_stamp: "Formatted at 2026-08-10 00:52:19 on dist-test-slave-jlzn"
server_key: "e19b2da0992148fe4aeedb0dc8bf2905"
server_key_iv: "e436461fdb47c1bd551835ea6450938b"
server_key_version: "encryptionkey@0"
I20260810 00:52:19.572744 22446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "cbde9124c7bb4285994cfc3e1314a80b"
format_stamp: "Formatted at 2026-08-10 00:52:19 on dist-test-slave-jlzn"
server_key: "e19b2da0992148fe4aeedb0dc8bf2905"
server_key_iv: "e436461fdb47c1bd551835ea6450938b"
server_key_version: "encryptionkey@0"
I20260810 00:52:19.577343 22446 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:52:19.580469 22468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:19.581619 22446 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:52:19.581768 22446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "cbde9124c7bb4285994cfc3e1314a80b"
format_stamp: "Formatted at 2026-08-10 00:52:19 on dist-test-slave-jlzn"
server_key: "e19b2da0992148fe4aeedb0dc8bf2905"
server_key_iv: "e436461fdb47c1bd551835ea6450938b"
server_key_version: "encryptionkey@0"
I20260810 00:52:19.581954 22446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:19.615655 22446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:19.619675 22446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:19.619938 22446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:19.638947 22446 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:42341
I20260810 00:52:19.638967 22556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:42341 every 8 connection(s)
I20260810 00:52:19.640251 22446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:52:19.643671 22557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:19.649783 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 22446
I20260810 00:52:19.650029 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:52:19.650859 10629 external_mini_cluster.cc:1428] Setting key cbb1078ab30b62d460c4f127e295032f
I20260810 00:52:19.654328 22557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b: Bootstrap starting.
I20260810 00:52:19.657593 22557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:19.658695 22557 log.cc:824] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:19.661397 22557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b: No bootstrap required, opened a new log
Aug 10 00:52:19 dist-test-slave-jlzn krb5kdc[22426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM
I20260810 00:52:19.669371 22557 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbde9124c7bb4285994cfc3e1314a80b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42341 } }
I20260810 00:52:19.669695 22557 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:19.669788 22557 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbde9124c7bb4285994cfc3e1314a80b, State: Initialized, Role: FOLLOWER
I20260810 00:52:19.670336 22557 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbde9124c7bb4285994cfc3e1314a80b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42341 } }
I20260810 00:52:19.670562 22557 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:52:19.670697 22557 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:52:19.670869 22557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:19.672812 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:19.652835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36534 (local address 127.10.97.126:42341)
0810 00:52:19.653549 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:19.653559 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:19.653600 (+ 41us) server_negotiation.cc:407] Connection header received
0810 00:52:19.655362 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:19.655383 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:19.655737 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:19.656083 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:19.657770 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:19.659104 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:19.660648 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:19.661271 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:19.661396 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:19.665689 (+ 4293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:19.665697 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:19.665709 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:19.665751 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:19.668885 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:19.669561 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:19.669567 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:19.669573 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:19.669786 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:19.670182 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:19.670188 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:19.670191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:19.670757 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:19.671037 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:19.671442 (+ 405us) server_negotiation.cc:299] Negotiation successful
0810 00:52:19.671862 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":346,"threads_started":1}
I20260810 00:52:19.673558 22557 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbde9124c7bb4285994cfc3e1314a80b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42341 } }
I20260810 00:52:19.674091 22557 leader_election.cc:302] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbde9124c7bb4285994cfc3e1314a80b; no voters:
I20260810 00:52:19.674691 22557 leader_election.cc:288] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:52:19.674780 22567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:52:19.675424 22567 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b [term 1 LEADER]: Becoming Leader. State: Replica: cbde9124c7bb4285994cfc3e1314a80b, State: Running, Role: LEADER
I20260810 00:52:19.676034 22567 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbde9124c7bb4285994cfc3e1314a80b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42341 } }
I20260810 00:52:19.677645 22557 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:52:19.678623 22568 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbde9124c7bb4285994cfc3e1314a80b. Latest consensus state: current_term: 1 leader_uuid: "cbde9124c7bb4285994cfc3e1314a80b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbde9124c7bb4285994cfc3e1314a80b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42341 } } }
I20260810 00:52:19.678592 22573 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbde9124c7bb4285994cfc3e1314a80b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbde9124c7bb4285994cfc3e1314a80b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42341 } } }
I20260810 00:52:19.679086 22573 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:19.679667 22568 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:19.680537 22576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:52:19.685804 22576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:52:19.695227 22576 catalog_manager.cc:1347] Generated new cluster ID: 0d3ef81ff861498ba83af63a619f8143
I20260810 00:52:19.695336 22576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:52:19.709884 22576 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:52:19.711531 22576 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:52:19.718403 22576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b: Generated new TSK 0
I20260810 00:52:19.719231 22576 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.65.keytab.
Entry for principal oryx/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.65.keytab.
I20260810 00:52:19.777366 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:42341
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.65.keytab
--principal=oryx/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:34233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:20.091578 22596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:20.091864 22596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:20.092010 22596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:20.098982 22596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:20.099077 22596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:20.099164 22596 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 00:52:20.099241 22596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:52:20.108469 22596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:34233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.65.keytab
--principal=oryx/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.97.126:42341
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:20.109580 22596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:20.110843 22596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:20.123010 22605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:20.123040 22611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:20.123303 22602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:20.124020 22596 server_base.cc:1034] running on GCE node
I20260810 00:52:20.124552 22596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:20.125377 22596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:20.126585 22596 hybrid_clock.cc:648] HybridClock initialized: now 1786323140126523 us; error 30 us; skew 500 ppm
I20260810 00:52:20.126849 22596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:20 dist-test-slave-jlzn krb5kdc[22426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323140, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:20.129810 22596 init.cc:377] Logged in from keytab as oryx/127.10.97.65@KRBTEST.COM (short username oryx)
I20260810 00:52:20.131110 22596 webserver.cc:460] Webserver started at http://127.10.97.65:39629/ using document root <none> and password file <none>
I20260810 00:52:20.131752 22596 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:20.131887 22596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:20.132203 22596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:20.134920 22596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "e98decf6d77e48ccb5c9c2c62cbced27"
format_stamp: "Formatted at 2026-08-10 00:52:20 on dist-test-slave-jlzn"
server_key: "29a2c65805e0e8c343516a22e48016d8"
server_key_iv: "4cf3dbdf70e82aef26d259757fca73c4"
server_key_version: "encryptionkey@0"
I20260810 00:52:20.135679 22596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "e98decf6d77e48ccb5c9c2c62cbced27"
format_stamp: "Formatted at 2026-08-10 00:52:20 on dist-test-slave-jlzn"
server_key: "29a2c65805e0e8c343516a22e48016d8"
server_key_iv: "4cf3dbdf70e82aef26d259757fca73c4"
server_key_version: "encryptionkey@0"
I20260810 00:52:20.140334 22596 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260810 00:52:20.143496 22618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:20.144685 22596 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:52:20.144832 22596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "e98decf6d77e48ccb5c9c2c62cbced27"
format_stamp: "Formatted at 2026-08-10 00:52:20 on dist-test-slave-jlzn"
server_key: "29a2c65805e0e8c343516a22e48016d8"
server_key_iv: "4cf3dbdf70e82aef26d259757fca73c4"
server_key_version: "encryptionkey@0"
I20260810 00:52:20.145043 22596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:20.168632 22596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:20.172241 22596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:20.172484 22596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:20.174916 22596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:20.175063 22596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 00:52:20.175218 22596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:20.175278 22596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 00:52:20 dist-test-slave-jlzn krb5kdc[22426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323140, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM
I20260810 00:52:20.197816 22632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:20.175222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:50679 (local address 127.10.97.126:42341)
0810 00:52:20.175712 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:20.175719 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:20.176850 (+ 1131us) server_negotiation.cc:407] Connection header received
0810 00:52:20.179440 (+ 2590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:20.179445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:20.179510 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:20.179601 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:20.181944 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:20.182800 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:20.185591 (+ 2791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:20.186116 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:20.186220 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:20.190513 (+ 4293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:20.190522 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:20.190527 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:20.190576 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:20.194001 (+ 3425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:20.194937 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:20.194942 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:20.194944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:20.195072 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:20.195707 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:20.195713 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:20.195717 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:20.196155 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:20.196308 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:20.197372 (+ 1064us) server_negotiation.cc:299] Negotiation successful
0810 00:52:20.197558 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":226,"threads_started":1}
I20260810 00:52:20.199265 22635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:20.175786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42341 (local address 127.10.97.65:50679)
0810 00:52:20.176676 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:20.176728 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:20.179139 (+ 2411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:20.179907 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:20.179918 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:20.180758 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:20.181732 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:20.181761 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:20.182984 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:20.182989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:20.185411 (+ 2422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:20.185429 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:20.186350 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:20.186357 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:20.186660 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:20.187325 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:20.187355 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:20.190247 (+ 2892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:20.194149 (+ 3902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:20.194158 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:20.194176 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:20.194766 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:20.195189 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:20.195195 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:20.195199 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:20.195521 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:20.196393 (+ 872us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:20.196401 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:20.196897 (+ 496us) client_negotiation.cc:769] Sending connection context
0810 00:52:20.197253 (+ 356us) client_negotiation.cc:240] Negotiation successful
0810 00:52:20.197817 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":316,"threads_started":1}
I20260810 00:52:20.217553 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:20.204493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42341 (local address 127.10.97.65:50547)
0810 00:52:20.205091 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:20.205107 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:20.205215 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:20.205506 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:20.205508 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:20.205888 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:20.206188 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:20.206199 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:20.207503 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:20.207506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:20.208846 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:20.208858 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:20.209904 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:20.209910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:20.210081 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:20.210589 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:20.210605 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:20.211247 (+ 642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:20.214720 (+ 3473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:20.214725 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:20.214732 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:20.215267 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:20.215695 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:20.215703 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:20.215707 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:20.215905 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:20.216423 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:20.216428 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:20.216640 (+ 212us) client_negotiation.cc:769] Sending connection context
0810 00:52:20.216816 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 00:52:20.217083 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":316,"threads_started":1}
I20260810 00:52:20.218397 22632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:20.204507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:50547 (local address 127.10.97.126:42341)
0810 00:52:20.204681 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:20.204688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:20.205178 (+ 490us) server_negotiation.cc:407] Connection header received
0810 00:52:20.205308 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:20.205314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:20.205390 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:20.205508 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:20.206326 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:20.207368 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:20.208986 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:20.209662 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:20.209792 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:20.211403 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:20.211409 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:20.211415 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:20.211449 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:20.214557 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:20.215379 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:20.215385 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:20.215388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:20.215547 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:20.216035 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:20.216040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:20.216044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:20.216289 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:20.216437 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:20.217862 (+ 1425us) server_negotiation.cc:299] Negotiation successful
0810 00:52:20.218084 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 00:52:20.227916 22596 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:34977
I20260810 00:52:20.227973 22797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:34977 every 8 connection(s)
I20260810 00:52:20.229061 22596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:52:20.230804 22798 heartbeater.cc:347] Connected to a master server at 127.10.97.126:42341
I20260810 00:52:20.231006 22798 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:20.231618 22798 heartbeater.cc:511] Master 127.10.97.126:42341 requested a full tablet report, sending...
I20260810 00:52:20.233474 22491 ts_manager.cc:194] Registered new tserver with Master: e98decf6d77e48ccb5c9c2c62cbced27 (127.10.97.65:34977)
I20260810 00:52:20.234851 22491 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:50679
I20260810 00:52:20.236959 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 22596
I20260810 00:52:20.237157 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:52:20.237738 10629 external_mini_cluster.cc:1428] Setting key 0388ec722fcac2e9697b4008ceaa3cf2
WARNING: no policy specified for oryx/127.10.97.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.97.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.66.keytab.
Entry for principal oryx/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.66.keytab.
I20260810 00:52:20.290290 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:42341
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.66.keytab
--principal=oryx/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:34233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:20.601354 22802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:20.601608 22802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:20.601708 22802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:20.608539 22802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:20.608637 22802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:20.608724 22802 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 00:52:20.608800 22802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:52:20.617820 22802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:34233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.66.keytab
--principal=oryx/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.97.126:42341
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:20.618878 22802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:20.620131 22802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:20.632355 22811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:20.632797 22810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:20.632772 22813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:20.633651 22802 server_base.cc:1034] running on GCE node
I20260810 00:52:20.634192 22802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:20.634958 22802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:20.636160 22802 hybrid_clock.cc:648] HybridClock initialized: now 1786323140636089 us; error 40 us; skew 500 ppm
I20260810 00:52:20.636428 22802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:20 dist-test-slave-jlzn krb5kdc[22426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323140, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:20.642913 22802 init.cc:377] Logged in from keytab as oryx/127.10.97.66@KRBTEST.COM (short username oryx)
I20260810 00:52:20.644371 22802 webserver.cc:460] Webserver started at http://127.10.97.66:40929/ using document root <none> and password file <none>
I20260810 00:52:20.645001 22802 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:20.645097 22802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:20.645393 22802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:20.648172 22802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "2955df32ef144c688056edf331c6bd99"
format_stamp: "Formatted at 2026-08-10 00:52:20 on dist-test-slave-jlzn"
server_key: "f231c694404e51442148adcf9e94db26"
server_key_iv: "7c3f88ff63ca9416f91c671c557b38a9"
server_key_version: "encryptionkey@0"
I20260810 00:52:20.648904 22802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "2955df32ef144c688056edf331c6bd99"
format_stamp: "Formatted at 2026-08-10 00:52:20 on dist-test-slave-jlzn"
server_key: "f231c694404e51442148adcf9e94db26"
server_key_iv: "7c3f88ff63ca9416f91c671c557b38a9"
server_key_version: "encryptionkey@0"
I20260810 00:52:20.653605 22802 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:52:20.656692 22826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:20.657835 22802 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:52:20.657971 22802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "2955df32ef144c688056edf331c6bd99"
format_stamp: "Formatted at 2026-08-10 00:52:20 on dist-test-slave-jlzn"
server_key: "f231c694404e51442148adcf9e94db26"
server_key_iv: "7c3f88ff63ca9416f91c671c557b38a9"
server_key_version: "encryptionkey@0"
I20260810 00:52:20.658162 22802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:20.671092 22802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:20.674484 22802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:20.674718 22802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:20.677156 22802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:20.677249 22802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:20.677387 22802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:20.677448 22802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 00:52:20 dist-test-slave-jlzn krb5kdc[22426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323140, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM
I20260810 00:52:20.697912 22632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:20.677449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:47363 (local address 127.10.97.126:42341)
0810 00:52:20.677651 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:20.677659 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:20.679070 (+ 1411us) server_negotiation.cc:407] Connection header received
0810 00:52:20.681165 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:20.681171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:20.681269 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:20.681355 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:20.683336 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:20.684468 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:20.686602 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:20.687281 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:20.687404 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:20.691752 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:20.691758 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:20.691761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:20.691793 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:20.694605 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:20.695461 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:20.695465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:20.695467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:20.695594 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:20.696190 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:20.696194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:20.696196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:20.696509 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:20.696626 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:20.697504 (+ 878us) server_negotiation.cc:299] Negotiation successful
0810 00:52:20.697673 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 00:52:20.699440 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:20.678042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42341 (local address 127.10.97.66:47363)
0810 00:52:20.678914 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:20.678956 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:20.680921 (+ 1965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:20.681614 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:20.681625 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:20.682329 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:20.683079 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:20.683102 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:20.684690 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:20.684696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:20.686447 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:20.686460 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:20.687499 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:20.687505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:20.687708 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:20.688490 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:20.688524 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:20.691533 (+ 3009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:20.694738 (+ 3205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:20.694747 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:20.694762 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:20.695358 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:20.695779 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:20.695784 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:20.695788 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:20.696079 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:20.696620 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:20.696627 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:20.697069 (+ 442us) client_negotiation.cc:769] Sending connection context
0810 00:52:20.697394 (+ 325us) client_negotiation.cc:240] Negotiation successful
0810 00:52:20.697902 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":351,"threads_started":1}
I20260810 00:52:20.718787 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:20.704825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42341 (local address 127.10.97.66:49495)
0810 00:52:20.705295 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:20.705309 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:20.705433 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:20.705770 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:20.705778 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:20.706565 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:20.707040 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:20.707055 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:20.708313 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:20.708318 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:20.709799 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:20.709814 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:20.710684 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:20.710691 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:20.710891 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:20.711445 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:20.711467 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:20.712621 (+ 1154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:20.715888 (+ 3267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:20.715894 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:20.715898 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:20.716443 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:20.716832 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:20.716838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:20.716842 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:20.716994 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:20.717720 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:20.717725 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:20.717890 (+ 165us) client_negotiation.cc:769] Sending connection context
0810 00:52:20.718014 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 00:52:20.718349 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":253,"threads_started":1}
I20260810 00:52:20.719089 22632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:20.704749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:49495 (local address 127.10.97.126:42341)
0810 00:52:20.704891 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:20.704894 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:20.705387 (+ 493us) server_negotiation.cc:407] Connection header received
0810 00:52:20.705561 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:20.705565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:20.705626 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:20.705718 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:20.707204 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:20.708132 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:20.709989 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:20.710477 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:20.710601 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:20.712819 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:20.712826 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:20.712830 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:20.712868 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:20.715705 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:20.716579 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:20.716583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:20.716585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:20.716716 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:20.717126 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:20.717132 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:20.717135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:20.717572 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:20.717715 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:20.718135 (+ 420us) server_negotiation.cc:299] Negotiation successful
0810 00:52:20.718326 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 00:52:20.726989 22802 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:42311
I20260810 00:52:20.727030 23002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:42311 every 8 connection(s)
I20260810 00:52:20.728124 22802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:52:20.729203 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 22802
I20260810 00:52:20.729401 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:52:20.729907 10629 external_mini_cluster.cc:1428] Setting key d81becbe6a647b6e0b6287e5b4bef10c
I20260810 00:52:20.729979 23004 heartbeater.cc:347] Connected to a master server at 127.10.97.126:42341
I20260810 00:52:20.730962 23004 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:20.731590 23004 heartbeater.cc:511] Master 127.10.97.126:42341 requested a full tablet report, sending...
I20260810 00:52:20.733068 22491 ts_manager.cc:194] Registered new tserver with Master: 2955df32ef144c688056edf331c6bd99 (127.10.97.66:42311)
I20260810 00:52:20.733826 22491 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:47363
WARNING: no policy specified for oryx/127.10.97.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.97.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.67.keytab.
Entry for principal oryx/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.67.keytab.
I20260810 00:52:20.780169 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:42341
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.67.keytab
--principal=oryx/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:34233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:21.097280 23010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:21.097579 23010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:21.097731 23010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:21.104719 23010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:21.104827 23010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:21.104944 23010 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 00:52:21.105048 23010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:52:21.114606 23010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:34233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.67.keytab
--principal=oryx/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.97.126:42341
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:21.115747 23010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:21.116995 23010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:21.129405 23017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:21.129333 23019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:21.129374 23022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:21.130100 23010 server_base.cc:1034] running on GCE node
I20260810 00:52:21.130606 23010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:21.131388 23010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:21.132583 23010 hybrid_clock.cc:648] HybridClock initialized: now 1786323141132514 us; error 42 us; skew 500 ppm
I20260810 00:52:21.132833 23010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:21 dist-test-slave-jlzn krb5kdc[22426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323141, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:21.135728 23010 init.cc:377] Logged in from keytab as oryx/127.10.97.67@KRBTEST.COM (short username oryx)
I20260810 00:52:21.136979 23010 webserver.cc:460] Webserver started at http://127.10.97.67:42063/ using document root <none> and password file <none>
I20260810 00:52:21.137564 23010 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:21.137646 23010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:21.137900 23010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:21.140620 23010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "855f3f1a72704b48a8bea54dc4a48f76"
format_stamp: "Formatted at 2026-08-10 00:52:21 on dist-test-slave-jlzn"
server_key: "4c287aa572d81bfc496384722809cd1f"
server_key_iv: "67a7556ce5079004cc6c0921f4b3c7ef"
server_key_version: "encryptionkey@0"
I20260810 00:52:21.141319 23010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "855f3f1a72704b48a8bea54dc4a48f76"
format_stamp: "Formatted at 2026-08-10 00:52:21 on dist-test-slave-jlzn"
server_key: "4c287aa572d81bfc496384722809cd1f"
server_key_iv: "67a7556ce5079004cc6c0921f4b3c7ef"
server_key_version: "encryptionkey@0"
I20260810 00:52:21.145936 23010 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:52:21.149060 23033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:21.150295 23010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:52:21.150418 23010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "855f3f1a72704b48a8bea54dc4a48f76"
format_stamp: "Formatted at 2026-08-10 00:52:21 on dist-test-slave-jlzn"
server_key: "4c287aa572d81bfc496384722809cd1f"
server_key_iv: "67a7556ce5079004cc6c0921f4b3c7ef"
server_key_version: "encryptionkey@0"
I20260810 00:52:21.150583 23010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:21.170322 23010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:21.173854 23010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:21.174082 23010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:21.176558 23010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:21.176638 23010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:21.176791 23010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:21.176873 23010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 00:52:21 dist-test-slave-jlzn krb5kdc[22426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323141, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM
I20260810 00:52:21.198594 22632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:21.176919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:44485 (local address 127.10.97.126:42341)
0810 00:52:21.177109 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:21.177117 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:21.178517 (+ 1400us) server_negotiation.cc:407] Connection header received
0810 00:52:21.180614 (+ 2097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:21.180620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:21.180709 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:21.180795 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:21.182691 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:21.183901 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:21.186182 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:21.186877 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:21.187011 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:21.191509 (+ 4498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:21.191518 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:21.191523 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:21.191568 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:21.195143 (+ 3575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:21.195844 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:21.195850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:21.195854 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:21.196032 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:21.196586 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:21.196592 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:21.196595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:21.196955 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:21.197079 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:21.198003 (+ 924us) server_negotiation.cc:299] Negotiation successful
0810 00:52:21.198235 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 00:52:21.200277 23044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:21.177471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42341 (local address 127.10.97.67:44485)
0810 00:52:21.178335 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:21.178381 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:21.180355 (+ 1974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:21.181066 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:21.181080 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:21.181779 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:21.182495 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:21.182518 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:21.184060 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:21.184066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:21.186016 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:21.186029 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:21.187132 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:21.187141 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:21.187385 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:21.188159 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:21.188196 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:21.191246 (+ 3050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:21.195309 (+ 4063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:21.195318 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:21.195332 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:21.195726 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:21.196180 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:21.196184 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:21.196186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:21.196423 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:21.197206 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:21.197214 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:21.197611 (+ 397us) client_negotiation.cc:769] Sending connection context
0810 00:52:21.197875 (+ 264us) client_negotiation.cc:240] Negotiation successful
0810 00:52:21.198407 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":399,"threads_started":1}
I20260810 00:52:21.213605 23010 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:35309
I20260810 00:52:21.213691 23198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:35309 every 8 connection(s)
I20260810 00:52:21.214782 23010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:52:21.216843 23199 heartbeater.cc:347] Connected to a master server at 127.10.97.126:42341
I20260810 00:52:21.217044 23199 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:21.217675 23199 heartbeater.cc:511] Master 127.10.97.126:42341 requested a full tablet report, sending...
I20260810 00:52:21.218928 22491 ts_manager.cc:194] Registered new tserver with Master: 855f3f1a72704b48a8bea54dc4a48f76 (127.10.97.67:35309)
I20260810 00:52:21.219538 23174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:21.206659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42341 (local address 127.10.97.67:39933)
0810 00:52:21.207239 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:21.207254 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:21.207380 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:21.207857 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:21.207862 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:21.208560 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:21.209004 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:21.209019 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:21.210259 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:21.210264 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:21.211756 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:21.211772 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:21.212572 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:21.212579 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:21.212726 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:21.213319 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:21.213336 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:21.214160 (+ 824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:21.217202 (+ 3042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:21.217206 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:21.217208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:21.217585 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:21.217952 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:21.217956 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:21.217958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:21.218075 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:21.218561 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:21.218565 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:21.218726 (+ 161us) client_negotiation.cc:769] Sending connection context
0810 00:52:21.218850 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 00:52:21.219125 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":318,"threads_started":1}
I20260810 00:52:21.219966 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 23010
I20260810 00:52:21.220008 22491 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:44485
I20260810 00:52:21.220165 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:52:21.219570 22632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:21.206555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:39933 (local address 127.10.97.126:42341)
0810 00:52:21.206705 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:21.206709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:21.207336 (+ 627us) server_negotiation.cc:407] Connection header received
0810 00:52:21.207490 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:21.207495 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:21.207568 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:21.207670 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:21.209200 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:21.210088 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:21.211964 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:21.212356 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:21.212467 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:21.214320 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:21.214325 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:21.214328 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:21.214357 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:21.217063 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:21.217691 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:21.217697 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:21.217701 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:21.217848 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:21.218177 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:21.218184 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:21.218188 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:21.218445 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:21.218574 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:21.218988 (+ 414us) server_negotiation.cc:299] Negotiation successful
0810 00:52:21.219199 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 00:52:21.220662 10629 external_mini_cluster.cc:1428] Setting key 6602508f58f231d66349ae580223e735
I20260810 00:52:21.223757 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:52:21.237251 22798 heartbeater.cc:503] Master 127.10.97.126:42341 was elected leader, sending a full tablet report...
Aug 10 00:52:21 dist-test-slave-jlzn krb5kdc[22426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323141, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 00:52:21 dist-test-slave-jlzn krb5kdc[22426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:21 dist-test-slave-jlzn krb5kdc[22426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:21.736333 23004 heartbeater.cc:503] Master 127.10.97.126:42341 was elected leader, sending a full tablet report...
I20260810 00:52:22.223059 23199 heartbeater.cc:503] Master 127.10.97.126:42341 was elected leader, sending a full tablet report...
W20260810 00:52:22.259308 23216 sasl_common.cc:323] Time spent calling sasl_client_start(): real 0.988s user 0.008s sys 0.008s
W20260810 00:52:22.260368 22632 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:21.266542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36544 (local address 127.10.97.126:42341)
0810 00:52:21.266769 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:21.266776 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:21.266984 (+ 208us) server_negotiation.cc:407] Connection header received
0810 00:52:21.267127 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:21.267131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:21.267205 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:21.267287 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:21.268523 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:21.269249 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:21.270364 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:21.270760 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:21.270844 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:22.260150 (+989306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36544: BlockingRecv error: recv got EOF from 127.0.0.1:36544 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 10 00:52:22 dist-test-slave-jlzn krb5kdc[22426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323141, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM
I20260810 00:52:22.276263 22632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:22.263851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48034 (local address 127.10.97.126:42341)
0810 00:52:22.264032 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:22.264038 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:22.264342 (+ 304us) server_negotiation.cc:407] Connection header received
0810 00:52:22.264514 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:22.264520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:22.264600 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:22.264694 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:22.265906 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.266663 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.267877 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.268295 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.268405 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:22.271977 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:22.271982 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:22.271986 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:22.272018 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:22.274249 (+ 2231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:22.274804 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:22.274807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:22.274810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:22.274917 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:22.275210 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:22.275212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:22.275214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:22.275468 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:22.275696 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:22.275883 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 00:52:22.276048 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 00:52:22.279841 22491 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:52:22.286597 22491 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:52:22.309059 23233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:22.298281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33072 (local address 127.10.97.67:35309)
0810 00:52:22.298870 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:22.298879 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:22.300142 (+ 1263us) server_negotiation.cc:407] Connection header received
0810 00:52:22.300429 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:22.300436 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:22.300680 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:22.300844 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:22.301870 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.303385 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.306201 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.307609 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.307719 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:22.307951 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:22.308582 (+ 631us) server_negotiation.cc:299] Negotiation successful
0810 00:52:22.308764 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":279,"threads_started":1}
I20260810 00:52:22.310060 23232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:22.297688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:35309 (local address 127.0.0.1:33072)
0810 00:52:22.300033 (+ 2345us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:22.300051 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:22.300253 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:22.301180 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:22.301187 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:22.301213 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:22.301662 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:22.301677 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.303668 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.303674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:22.306003 (+ 2329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:22.306018 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.308139 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.308145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:22.308324 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:22.308380 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 00:52:22.309525 (+ 1145us) client_negotiation.cc:240] Negotiation successful
0810 00:52:22.309734 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2212,"thread_start_us":2058,"threads_started":1}
I20260810 00:52:22.311077 23230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:22.297521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:42311 (local address 127.0.0.1:53484)
0810 00:52:22.299421 (+ 1900us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:22.299457 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:22.299602 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:22.301986 (+ 2384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:22.301991 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:22.302016 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:22.302466 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:22.302481 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.305491 (+ 3010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.305496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:22.307601 (+ 2105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:22.307615 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.310521 (+ 2906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.310527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:22.310709 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:22.310732 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 00:52:22.310777 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 00:52:22.310857 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1691,"thread_start_us":1514,"threads_started":1}
I20260810 00:52:22.312033 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:22.297682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53484 (local address 127.10.97.66:42311)
0810 00:52:22.300202 (+ 2520us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:22.300210 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:22.300232 (+ 22us) server_negotiation.cc:407] Connection header received
0810 00:52:22.300322 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:22.300329 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:22.301792 (+ 1463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:22.302651 (+ 859us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:22.302828 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.305347 (+ 2519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.307822 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.310290 (+ 2468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.311308 (+ 1018us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:22.311522 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:22.311588 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 00:52:22.311741 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2364,"thread_start_us":2213,"threads_started":1}
I20260810 00:52:22.315608 23106 tablet_service.cc:1467] Processing CreateTablet for tablet 75e8a00581614090b37b7501a8d4c40d (DEFAULT_TABLE table=test-table [id=cf9db021847e4a00bf5fcd6b6dd21d84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:22.315609 22903 tablet_service.cc:1467] Processing CreateTablet for tablet 75e8a00581614090b37b7501a8d4c40d (DEFAULT_TABLE table=test-table [id=cf9db021847e4a00bf5fcd6b6dd21d84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:22.317107 22903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75e8a00581614090b37b7501a8d4c40d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:22.317106 23106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75e8a00581614090b37b7501a8d4c40d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:22.317664 23232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:22.300101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34977 (local address 127.0.0.1:50086)
0810 00:52:22.310461 (+ 10360us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:22.310478 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:22.310565 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:22.312808 (+ 2243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:22.312813 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:22.312836 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:22.312991 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:22.313002 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.314570 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.314574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:22.315663 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:22.315675 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.316976 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.316982 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:22.317165 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:22.317196 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 00:52:22.317284 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 00:52:22.317411 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10268,"thread_start_us":11605,"threads_started":1}
I20260810 00:52:22.317737 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:22.300434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50086 (local address 127.10.97.65:34977)
0810 00:52:22.312364 (+ 11930us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:22.312371 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:22.312393 (+ 22us) server_negotiation.cc:407] Connection header received
0810 00:52:22.312464 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:22.312469 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:22.312637 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:22.312793 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:22.313187 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.314436 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.315877 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.316821 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.316943 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:22.317149 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:22.317335 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 00:52:22.317479 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11787,"thread_start_us":11563,"threads_started":1}
I20260810 00:52:22.320863 22695 tablet_service.cc:1467] Processing CreateTablet for tablet 75e8a00581614090b37b7501a8d4c40d (DEFAULT_TABLE table=test-table [id=cf9db021847e4a00bf5fcd6b6dd21d84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:22.322005 22695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75e8a00581614090b37b7501a8d4c40d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:22.332448 23241 tablet_bootstrap.cc:492] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27: Bootstrap starting.
I20260810 00:52:22.334280 23239 tablet_bootstrap.cc:492] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99: Bootstrap starting.
I20260810 00:52:22.335018 23238 tablet_bootstrap.cc:492] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76: Bootstrap starting.
I20260810 00:52:22.336390 23241 tablet_bootstrap.cc:654] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:22.337358 23241 log.cc:824] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:22.338033 23239 tablet_bootstrap.cc:654] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:22.339215 23239 log.cc:824] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:22.339694 23238 tablet_bootstrap.cc:654] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:22.339926 23241 tablet_bootstrap.cc:492] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27: No bootstrap required, opened a new log
I20260810 00:52:22.340205 23241 ts_tablet_manager.cc:1397] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260810 00:52:22.340798 23238 log.cc:824] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:22.341964 23239 tablet_bootstrap.cc:492] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99: No bootstrap required, opened a new log
I20260810 00:52:22.342227 23239 ts_tablet_manager.cc:1397] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260810 00:52:22.343605 23238 tablet_bootstrap.cc:492] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76: No bootstrap required, opened a new log
I20260810 00:52:22.343888 23238 ts_tablet_manager.cc:1397] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 00:52:22.347582 23241 raft_consensus.cc:348] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } } peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } }
I20260810 00:52:22.347946 23241 raft_consensus.cc:374] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:22.348086 23241 raft_consensus.cc:729] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e98decf6d77e48ccb5c9c2c62cbced27, State: Initialized, Role: FOLLOWER
I20260810 00:52:22.348686 23241 consensus_queue.cc:260] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } } peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } }
I20260810 00:52:22.350193 23241 ts_tablet_manager.cc:1428] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27: Time spent starting tablet: real 0.010s user 0.003s sys 0.007s
I20260810 00:52:22.353400 23239 raft_consensus.cc:348] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } } peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } }
I20260810 00:52:22.353816 23239 raft_consensus.cc:374] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:22.353668 23238 raft_consensus.cc:348] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } } peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } }
I20260810 00:52:22.353955 23239 raft_consensus.cc:729] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2955df32ef144c688056edf331c6bd99, State: Initialized, Role: FOLLOWER
I20260810 00:52:22.354058 23238 raft_consensus.cc:374] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:22.354162 23238 raft_consensus.cc:729] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 855f3f1a72704b48a8bea54dc4a48f76, State: Initialized, Role: FOLLOWER
I20260810 00:52:22.354647 23239 consensus_queue.cc:260] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } } peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } }
I20260810 00:52:22.354660 23238 consensus_queue.cc:260] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } } peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } }
I20260810 00:52:22.356253 23239 ts_tablet_manager.cc:1428] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260810 00:52:22.356258 23238 ts_tablet_manager.cc:1428] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
W20260810 00:52:22.467252 23201 tablet.cc:2369] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:52:22.480386 23007 tablet.cc:2369] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:52:22.481683 22799 tablet.cc:2369] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:52:22.571587 23251 raft_consensus.cc:482] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:52:22.571925 23251 raft_consensus.cc:504] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } } peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } }
I20260810 00:52:22.573549 23251 leader_election.cc:288] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2955df32ef144c688056edf331c6bd99 (127.10.97.66:42311), e98decf6d77e48ccb5c9c2c62cbced27 (127.10.97.65:34977)
I20260810 00:52:22.582335 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:22.574155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:56069 (local address 127.10.97.65:34977)
0810 00:52:22.574341 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:22.574346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:22.574858 (+ 512us) server_negotiation.cc:407] Connection header received
0810 00:52:22.574985 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:22.575012 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:22.575082 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:22.575185 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:22.575977 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.577077 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.580052 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.581279 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.581442 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:22.581512 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:22.581952 (+ 440us) server_negotiation.cc:299] Negotiation successful
0810 00:52:22.582083 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 00:52:22.582332 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:22.573907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:42073 (local address 127.10.97.66:42311)
0810 00:52:22.574104 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:22.574110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:22.574780 (+ 670us) server_negotiation.cc:407] Connection header received
0810 00:52:22.574994 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:22.574998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:22.575082 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:22.575185 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:22.576018 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.577077 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.580053 (+ 2976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.581282 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.581437 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:22.581512 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:22.581948 (+ 436us) server_negotiation.cc:299] Negotiation successful
0810 00:52:22.582082 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 00:52:22.582576 23254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:22.573964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34977 (local address 127.10.97.67:56069)
0810 00:52:22.574756 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:22.574771 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:22.574863 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:22.575206 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:22.575211 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:22.575266 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:22.575782 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:22.575825 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.577248 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.577252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:22.579890 (+ 2638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:22.579901 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.581538 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.581544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:22.581748 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:22.581806 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 00:52:22.581898 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 00:52:22.582075 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":161,"spinlock_wait_cycles":8832,"thread_start_us":546,"threads_started":1}
I20260810 00:52:22.582597 23253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:22.573888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:42311 (local address 127.10.97.67:42073)
0810 00:52:22.574663 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:22.574682 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:22.574844 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:22.575206 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:22.575211 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:22.575264 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:22.575890 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:22.575901 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.577246 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.577252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:22.579883 (+ 2631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:22.579901 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.581514 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.581526 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:22.581751 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:22.581812 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 00:52:22.581898 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 00:52:22.582073 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"mutex_wait_us":287,"spinlock_wait_cycles":33408,"thread_start_us":388,"threads_started":1}
I20260810 00:52:22.583458 22935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e8a00581614090b37b7501a8d4c40d" candidate_uuid: "855f3f1a72704b48a8bea54dc4a48f76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2955df32ef144c688056edf331c6bd99" is_pre_election: true
I20260810 00:52:22.583648 22733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e8a00581614090b37b7501a8d4c40d" candidate_uuid: "855f3f1a72704b48a8bea54dc4a48f76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" is_pre_election: true
I20260810 00:52:22.583977 22935 raft_consensus.cc:2454] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 855f3f1a72704b48a8bea54dc4a48f76 in term 0.
I20260810 00:52:22.584178 22733 raft_consensus.cc:2454] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 855f3f1a72704b48a8bea54dc4a48f76 in term 0.
I20260810 00:52:22.584642 23037 leader_election.cc:302] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2955df32ef144c688056edf331c6bd99, 855f3f1a72704b48a8bea54dc4a48f76; no voters:
I20260810 00:52:22.585052 23251 raft_consensus.cc:2781] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:52:22.585211 23251 raft_consensus.cc:482] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:52:22.585316 23251 raft_consensus.cc:3037] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:22.587672 23251 raft_consensus.cc:504] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } } peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } }
I20260810 00:52:22.588258 23251 leader_election.cc:288] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [CANDIDATE]: Term 1 election: Requested vote from peers 2955df32ef144c688056edf331c6bd99 (127.10.97.66:42311), e98decf6d77e48ccb5c9c2c62cbced27 (127.10.97.65:34977)
I20260810 00:52:22.588624 22935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e8a00581614090b37b7501a8d4c40d" candidate_uuid: "855f3f1a72704b48a8bea54dc4a48f76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2955df32ef144c688056edf331c6bd99"
I20260810 00:52:22.588704 22733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e8a00581614090b37b7501a8d4c40d" candidate_uuid: "855f3f1a72704b48a8bea54dc4a48f76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98decf6d77e48ccb5c9c2c62cbced27"
I20260810 00:52:22.588802 22935 raft_consensus.cc:3037] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:22.588869 22733 raft_consensus.cc:3037] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:22.590893 22935 raft_consensus.cc:2454] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 855f3f1a72704b48a8bea54dc4a48f76 in term 1.
I20260810 00:52:22.590912 22733 raft_consensus.cc:2454] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 855f3f1a72704b48a8bea54dc4a48f76 in term 1.
I20260810 00:52:22.591246 23038 leader_election.cc:302] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 855f3f1a72704b48a8bea54dc4a48f76, e98decf6d77e48ccb5c9c2c62cbced27; no voters:
I20260810 00:52:22.591547 23251 raft_consensus.cc:2781] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:52:22.591954 23251 raft_consensus.cc:686] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [term 1 LEADER]: Becoming Leader. State: Replica: 855f3f1a72704b48a8bea54dc4a48f76, State: Running, Role: LEADER
I20260810 00:52:22.592435 23251 consensus_queue.cc:237] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } } peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } }
I20260810 00:52:22.596956 22489 catalog_manager.cc:5495] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 855f3f1a72704b48a8bea54dc4a48f76 (127.10.97.67). New cstate: current_term: 1 leader_uuid: "855f3f1a72704b48a8bea54dc4a48f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } health_report { overall_health: UNKNOWN } } }
I20260810 00:52:22.668499 23233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:22.663390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33086 (local address 127.10.97.67:35309)
0810 00:52:22.663594 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:22.663599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:22.663677 (+ 78us) server_negotiation.cc:407] Connection header received
0810 00:52:22.663918 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:22.663922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:22.664000 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:22.664141 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:52:22.664487 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.665191 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.666500 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:22.666896 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:22.667005 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:22.667410 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:52:22.667956 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:52:22.668094 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:22.668171 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 00:52:22.668252 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 00:52:22.675993 22733 raft_consensus.cc:1261] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27 [term 1 FOLLOWER]: Refusing update from remote peer 855f3f1a72704b48a8bea54dc4a48f76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:52:22.676033 22935 raft_consensus.cc:1261] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99 [term 1 FOLLOWER]: Refusing update from remote peer 855f3f1a72704b48a8bea54dc4a48f76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:52:22.676896 23259 consensus_queue.cc:1035] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:52:22.677253 23251 consensus_queue.cc:1035] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:52:22.688089 23263 mvcc.cc:205] Tried to move back new op lower bound from 7316779592391630848 to 7316779592061915136. Current Snapshot: MvccSnapshot[applied={T|T < 7316779592391630848}]
I20260810 00:52:22.700233 22491 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:52:22.700491 22491 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:52:22.703969 22491 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b: Sending DeleteTablet for 3 replicas of tablet 75e8a00581614090b37b7501a8d4c40d
I20260810 00:52:22.704921 22903 tablet_service.cc:1514] Processing DeleteTablet for tablet 75e8a00581614090b37b7501a8d4c40d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:22 UTC) from {username='oryx', principal='oryx/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:53484
I20260810 00:52:22.705052 23106 tablet_service.cc:1514] Processing DeleteTablet for tablet 75e8a00581614090b37b7501a8d4c40d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:22 UTC) from {username='oryx', principal='oryx/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:33072
I20260810 00:52:22.705554 22695 tablet_service.cc:1514] Processing DeleteTablet for tablet 75e8a00581614090b37b7501a8d4c40d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:22 UTC) from {username='oryx', principal='oryx/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:50086
I20260810 00:52:22.705729 23280 tablet_replica.cc:330] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99: stopping tablet replica
I20260810 00:52:22.705852 23282 tablet_replica.cc:330] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76: stopping tablet replica
I20260810 00:52:22.706095 23280 raft_consensus.cc:2229] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:52:22.706221 23282 raft_consensus.cc:2229] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:52:22.706640 23280 raft_consensus.cc:2258] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:22.706867 23282 raft_consensus.cc:2258] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:22.707211 22491 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:52:22.707839 22491 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:52:22.708918 23280 ts_tablet_manager.cc:1905] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:22.708933 23282 ts_tablet_manager.cc:1905] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:22.715584 23280 ts_tablet_manager.cc:1918] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:52:22.715592 23282 ts_tablet_manager.cc:1918] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:52:22.715595 22692 tablet_service.cc:1467] Processing CreateTablet for tablet e64ae84fd07d44d09fabdd589af0b2bb (DEFAULT_TABLE table=test-table [id=b3f17bebca0e43c0bd79bcefb6f1e88e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:22.715758 23282 log.cc:1196] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/75e8a00581614090b37b7501a8d4c40d
I20260810 00:52:22.715788 23280 log.cc:1196] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/75e8a00581614090b37b7501a8d4c40d
I20260810 00:52:22.715615 23106 tablet_service.cc:1467] Processing CreateTablet for tablet e64ae84fd07d44d09fabdd589af0b2bb (DEFAULT_TABLE table=test-table [id=b3f17bebca0e43c0bd79bcefb6f1e88e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:22.716136 22692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e64ae84fd07d44d09fabdd589af0b2bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:22.716218 23106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e64ae84fd07d44d09fabdd589af0b2bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:22.716336 23282 ts_tablet_manager.cc:1939] T 75e8a00581614090b37b7501a8d4c40d P 855f3f1a72704b48a8bea54dc4a48f76: Deleting consensus metadata
I20260810 00:52:22.716346 23280 ts_tablet_manager.cc:1939] T 75e8a00581614090b37b7501a8d4c40d P 2955df32ef144c688056edf331c6bd99: Deleting consensus metadata
I20260810 00:52:22.720151 22471 catalog_manager.cc:4841] TS 2955df32ef144c688056edf331c6bd99 (127.10.97.66:42311): tablet 75e8a00581614090b37b7501a8d4c40d (table test-table [id=cf9db021847e4a00bf5fcd6b6dd21d84]) successfully deleted
I20260810 00:52:22.720476 22472 catalog_manager.cc:4841] TS 855f3f1a72704b48a8bea54dc4a48f76 (127.10.97.67:35309): tablet 75e8a00581614090b37b7501a8d4c40d (table test-table [id=cf9db021847e4a00bf5fcd6b6dd21d84]) successfully deleted
I20260810 00:52:22.720494 22903 tablet_service.cc:1467] Processing CreateTablet for tablet e64ae84fd07d44d09fabdd589af0b2bb (DEFAULT_TABLE table=test-table [id=b3f17bebca0e43c0bd79bcefb6f1e88e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:22.720934 22903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e64ae84fd07d44d09fabdd589af0b2bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:22.721864 23238 tablet_bootstrap.cc:492] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76: Bootstrap starting.
I20260810 00:52:22.723435 23238 tablet_bootstrap.cc:654] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:22.728988 23283 tablet_replica.cc:330] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27: stopping tablet replica
I20260810 00:52:22.729215 23241 tablet_bootstrap.cc:492] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27: Bootstrap starting.
I20260810 00:52:22.729477 23283 raft_consensus.cc:2229] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:52:22.729947 23283 raft_consensus.cc:2258] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:22.730841 23241 tablet_bootstrap.cc:654] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:22.731123 23239 tablet_bootstrap.cc:492] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99: Bootstrap starting.
I20260810 00:52:22.732445 23283 ts_tablet_manager.cc:1905] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:22.732831 23239 tablet_bootstrap.cc:654] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:22.741940 23283 ts_tablet_manager.cc:1918] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:52:22.744988 23239 tablet_bootstrap.cc:492] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99: No bootstrap required, opened a new log
I20260810 00:52:22.744990 23238 tablet_bootstrap.cc:492] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76: No bootstrap required, opened a new log
I20260810 00:52:22.745148 23239 ts_tablet_manager.cc:1397] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.007s
I20260810 00:52:22.745285 23238 ts_tablet_manager.cc:1397] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76: Time spent bootstrapping tablet: real 0.023s user 0.008s sys 0.005s
I20260810 00:52:22.746008 23239 raft_consensus.cc:348] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:22.746265 23239 raft_consensus.cc:374] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:22.746384 23239 raft_consensus.cc:729] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2955df32ef144c688056edf331c6bd99, State: Initialized, Role: FOLLOWER
I20260810 00:52:22.746183 23238 raft_consensus.cc:348] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:22.746764 23238 raft_consensus.cc:374] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:22.746701 23239 consensus_queue.cc:260] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:22.746956 23238 raft_consensus.cc:729] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 855f3f1a72704b48a8bea54dc4a48f76, State: Initialized, Role: FOLLOWER
I20260810 00:52:22.747280 23238 consensus_queue.cc:260] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:22.750298 23238 ts_tablet_manager.cc:1428] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260810 00:52:22.750497 23239 ts_tablet_manager.cc:1428] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260810 00:52:22.752473 23283 log.cc:1196] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/75e8a00581614090b37b7501a8d4c40d
I20260810 00:52:22.744257 23241 tablet_bootstrap.cc:492] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27: No bootstrap required, opened a new log
I20260810 00:52:22.752790 23241 ts_tablet_manager.cc:1397] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.007s
I20260810 00:52:22.753172 23283 ts_tablet_manager.cc:1939] T 75e8a00581614090b37b7501a8d4c40d P e98decf6d77e48ccb5c9c2c62cbced27: Deleting consensus metadata
I20260810 00:52:22.753733 23241 raft_consensus.cc:348] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:22.753959 23241 raft_consensus.cc:374] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:22.754066 23241 raft_consensus.cc:729] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e98decf6d77e48ccb5c9c2c62cbced27, State: Initialized, Role: FOLLOWER
I20260810 00:52:22.754335 23241 consensus_queue.cc:260] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:22.754918 23241 ts_tablet_manager.cc:1428] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 00:52:22.755386 22472 catalog_manager.cc:4841] TS e98decf6d77e48ccb5c9c2c62cbced27 (127.10.97.65:34977): tablet 75e8a00581614090b37b7501a8d4c40d (table test-table [id=cf9db021847e4a00bf5fcd6b6dd21d84]) successfully deleted
I20260810 00:52:23.088860 23252 raft_consensus.cc:482] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:52:23.089087 23252 raft_consensus.cc:504] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.090839 23252 leader_election.cc:288] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e98decf6d77e48ccb5c9c2c62cbced27 (127.10.97.65:34977), 855f3f1a72704b48a8bea54dc4a48f76 (127.10.97.67:35309)
I20260810 00:52:23.099227 23233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:23.092753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:55003 (local address 127.10.97.67:35309)
0810 00:52:23.092936 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:23.092942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:23.093195 (+ 253us) server_negotiation.cc:407] Connection header received
0810 00:52:23.093521 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:23.093527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:23.093604 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:23.093689 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:23.094385 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.095252 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.097042 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.098231 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.098398 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.098472 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:23.098777 (+ 305us) server_negotiation.cc:299] Negotiation successful
0810 00:52:23.098912 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 00:52:23.099341 23289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:23.091161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34977 (local address 127.10.97.66:40299)
0810 00:52:23.092187 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:23.092209 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:23.092346 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:23.093184 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:23.093189 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:23.093227 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:23.093569 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:23.093579 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.094776 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.094781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:23.096945 (+ 2164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:23.096961 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.098526 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.098531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:23.098656 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.098678 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 00:52:23.098738 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 00:52:23.098902 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"mutex_wait_us":50,"spinlock_wait_cycles":28800,"thread_start_us":770,"threads_started":1}
I20260810 00:52:23.099685 23291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:23.092358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:35309 (local address 127.10.97.66:55003)
0810 00:52:23.093101 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:23.093118 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:23.093352 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:23.093856 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:23.093861 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:23.093888 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:23.094226 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:23.094237 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.095376 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.095381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:23.096886 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:23.096898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.098380 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.098386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:23.098595 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.098632 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 00:52:23.098733 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 00:52:23.098883 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":29,"spinlock_wait_cycles":896,"thread_start_us":578,"threads_started":1}
I20260810 00:52:23.100100 23290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:23.091199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:40299 (local address 127.10.97.65:34977)
0810 00:52:23.092812 (+ 1613us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:23.092818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:23.092837 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:52:23.092905 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:23.092910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:23.092990 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:23.093133 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:23.093733 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.094645 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.097115 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.098281 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.099559 (+ 1278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.099624 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:23.099674 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 00:52:23.099777 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1491,"thread_start_us":1398,"threads_started":1}
I20260810 00:52:23.100605 23134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e64ae84fd07d44d09fabdd589af0b2bb" candidate_uuid: "2955df32ef144c688056edf331c6bd99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "855f3f1a72704b48a8bea54dc4a48f76" is_pre_election: true
I20260810 00:52:23.100695 22733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e64ae84fd07d44d09fabdd589af0b2bb" candidate_uuid: "2955df32ef144c688056edf331c6bd99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" is_pre_election: true
I20260810 00:52:23.100942 22733 raft_consensus.cc:2454] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2955df32ef144c688056edf331c6bd99 in term 0.
I20260810 00:52:23.100960 23134 raft_consensus.cc:2454] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2955df32ef144c688056edf331c6bd99 in term 0.
I20260810 00:52:23.101464 22830 leader_election.cc:302] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2955df32ef144c688056edf331c6bd99, e98decf6d77e48ccb5c9c2c62cbced27; no voters:
I20260810 00:52:23.101819 23252 raft_consensus.cc:2781] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:52:23.101974 23252 raft_consensus.cc:482] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:52:23.102075 23252 raft_consensus.cc:3037] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:23.105139 23252 raft_consensus.cc:504] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.105748 23252 leader_election.cc:288] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [CANDIDATE]: Term 1 election: Requested vote from peers e98decf6d77e48ccb5c9c2c62cbced27 (127.10.97.65:34977), 855f3f1a72704b48a8bea54dc4a48f76 (127.10.97.67:35309)
I20260810 00:52:23.106153 22733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e64ae84fd07d44d09fabdd589af0b2bb" candidate_uuid: "2955df32ef144c688056edf331c6bd99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98decf6d77e48ccb5c9c2c62cbced27"
I20260810 00:52:23.106187 23134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e64ae84fd07d44d09fabdd589af0b2bb" candidate_uuid: "2955df32ef144c688056edf331c6bd99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "855f3f1a72704b48a8bea54dc4a48f76"
I20260810 00:52:23.106335 22733 raft_consensus.cc:3037] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:23.106364 23134 raft_consensus.cc:3037] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:23.108490 23134 raft_consensus.cc:2454] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2955df32ef144c688056edf331c6bd99 in term 1.
I20260810 00:52:23.108868 22830 leader_election.cc:302] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2955df32ef144c688056edf331c6bd99, 855f3f1a72704b48a8bea54dc4a48f76; no voters:
I20260810 00:52:23.109094 23252 raft_consensus.cc:2781] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:52:23.109172 22733 raft_consensus.cc:2454] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2955df32ef144c688056edf331c6bd99 in term 1.
I20260810 00:52:23.109532 23252 raft_consensus.cc:686] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [term 1 LEADER]: Becoming Leader. State: Replica: 2955df32ef144c688056edf331c6bd99, State: Running, Role: LEADER
I20260810 00:52:23.110170 23252 consensus_queue.cc:237] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.113343 22491 catalog_manager.cc:5495] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2955df32ef144c688056edf331c6bd99 (127.10.97.66). New cstate: current_term: 1 leader_uuid: "2955df32ef144c688056edf331c6bd99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } health_report { overall_health: UNKNOWN } } }
Aug 10 00:52:23 dist-test-slave-jlzn krb5kdc[22426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323139, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.126@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM
I20260810 00:52:23.228919 23303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:23.214113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42341 (local address 127.0.0.1:48050)
0810 00:52:23.214921 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:23.214941 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:23.215088 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:23.215431 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:23.215435 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:23.216160 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:23.216421 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:23.216431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.217886 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.217891 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:23.218882 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:23.218895 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.219852 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.219857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:23.219977 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.220579 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:23.220597 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:23.222648 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:23.226224 (+ 3576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:23.226234 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:23.226240 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:23.226665 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:23.227025 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:23.227028 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:23.227031 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:23.227155 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:23.227817 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:23.227827 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:23.228004 (+ 177us) client_negotiation.cc:769] Sending connection context
0810 00:52:23.228175 (+ 171us) client_negotiation.cc:240] Negotiation successful
0810 00:52:23.228442 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":517,"threads_started":1}
I20260810 00:52:23.229317 23302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:23.214072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48050 (local address 127.10.97.126:42341)
0810 00:52:23.214695 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:23.214701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:23.215028 (+ 327us) server_negotiation.cc:407] Connection header received
0810 00:52:23.215203 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:23.215209 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:23.215286 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:23.215405 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:23.216571 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.217740 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.219010 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.219681 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.219835 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.222859 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:23.222866 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:23.222871 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:23.222905 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:23.226085 (+ 3180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:23.226772 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:23.226778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:23.226782 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:23.226923 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:23.227233 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:23.227238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:23.227241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:23.227551 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:23.227688 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:23.228284 (+ 596us) server_negotiation.cc:299] Negotiation successful
0810 00:52:23.228599 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":464,"thread_start_us":262,"threads_started":1}
I20260810 00:52:23.235172 22491 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:48050:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 00:52:23.236173 22491 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:52:23.242563 22692 tablet_service.cc:1467] Processing CreateTablet for tablet e007fc09469847e0b19485767945f735 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=538feb6e22b741eaa25c0619a9e63fd0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 00:52:23.242909 22903 tablet_service.cc:1467] Processing CreateTablet for tablet e007fc09469847e0b19485767945f735 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=538feb6e22b741eaa25c0619a9e63fd0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 00:52:23.243063 22692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e007fc09469847e0b19485767945f735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:23.243317 22903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e007fc09469847e0b19485767945f735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:23.248409 23106 tablet_service.cc:1467] Processing CreateTablet for tablet e007fc09469847e0b19485767945f735 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=538feb6e22b741eaa25c0619a9e63fd0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 00:52:23.248921 23106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e007fc09469847e0b19485767945f735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:23.248934 23239 tablet_bootstrap.cc:492] T e007fc09469847e0b19485767945f735 P 2955df32ef144c688056edf331c6bd99: Bootstrap starting.
I20260810 00:52:23.249894 23241 tablet_bootstrap.cc:492] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27: Bootstrap starting.
I20260810 00:52:23.250419 23239 tablet_bootstrap.cc:654] T e007fc09469847e0b19485767945f735 P 2955df32ef144c688056edf331c6bd99: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:23.251401 23241 tablet_bootstrap.cc:654] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:23.253412 23241 tablet_bootstrap.cc:492] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27: No bootstrap required, opened a new log
I20260810 00:52:23.253554 23241 ts_tablet_manager.cc:1397] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 00:52:23.253811 23239 tablet_bootstrap.cc:492] T e007fc09469847e0b19485767945f735 P 2955df32ef144c688056edf331c6bd99: No bootstrap required, opened a new log
I20260810 00:52:23.253957 23239 ts_tablet_manager.cc:1397] T e007fc09469847e0b19485767945f735 P 2955df32ef144c688056edf331c6bd99: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260810 00:52:23.254248 23241 raft_consensus.cc:348] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.254485 23241 raft_consensus.cc:374] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:23.254561 23241 raft_consensus.cc:729] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e98decf6d77e48ccb5c9c2c62cbced27, State: Initialized, Role: FOLLOWER
I20260810 00:52:23.254616 23306 tablet_bootstrap.cc:492] T e007fc09469847e0b19485767945f735 P 855f3f1a72704b48a8bea54dc4a48f76: Bootstrap starting.
I20260810 00:52:23.254680 23239 raft_consensus.cc:348] T e007fc09469847e0b19485767945f735 P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.254886 23239 raft_consensus.cc:374] T e007fc09469847e0b19485767945f735 P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:23.254809 23241 consensus_queue.cc:260] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.254951 23239 raft_consensus.cc:729] T e007fc09469847e0b19485767945f735 P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2955df32ef144c688056edf331c6bd99, State: Initialized, Role: FOLLOWER
I20260810 00:52:23.255218 23239 consensus_queue.cc:260] T e007fc09469847e0b19485767945f735 P 2955df32ef144c688056edf331c6bd99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.255821 23241 ts_tablet_manager.cc:1428] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 00:52:23.256335 23306 tablet_bootstrap.cc:654] T e007fc09469847e0b19485767945f735 P 855f3f1a72704b48a8bea54dc4a48f76: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:23.256335 23239 ts_tablet_manager.cc:1428] T e007fc09469847e0b19485767945f735 P 2955df32ef144c688056edf331c6bd99: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 00:52:23.255659 23311 tablet_replica.cc:439] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } } }
I20260810 00:52:23.256601 23252 tablet_replica.cc:439] T e007fc09469847e0b19485767945f735 P 2955df32ef144c688056edf331c6bd99: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } } }
I20260810 00:52:23.256767 23311 tablet_replica.cc:442] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:52:23.256868 23252 tablet_replica.cc:442] T e007fc09469847e0b19485767945f735 P 2955df32ef144c688056edf331c6bd99: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:52:23.258314 23306 tablet_bootstrap.cc:492] T e007fc09469847e0b19485767945f735 P 855f3f1a72704b48a8bea54dc4a48f76: No bootstrap required, opened a new log
I20260810 00:52:23.258502 23306 ts_tablet_manager.cc:1397] T e007fc09469847e0b19485767945f735 P 855f3f1a72704b48a8bea54dc4a48f76: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 00:52:23.259191 23306 raft_consensus.cc:348] T e007fc09469847e0b19485767945f735 P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.259421 23306 raft_consensus.cc:374] T e007fc09469847e0b19485767945f735 P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:23.259531 23306 raft_consensus.cc:729] T e007fc09469847e0b19485767945f735 P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 855f3f1a72704b48a8bea54dc4a48f76, State: Initialized, Role: FOLLOWER
I20260810 00:52:23.259750 23306 consensus_queue.cc:260] T e007fc09469847e0b19485767945f735 P 855f3f1a72704b48a8bea54dc4a48f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.260468 23306 ts_tablet_manager.cc:1428] T e007fc09469847e0b19485767945f735 P 855f3f1a72704b48a8bea54dc4a48f76: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 00:52:23.260414 23314 tablet_replica.cc:439] T e007fc09469847e0b19485767945f735 P 855f3f1a72704b48a8bea54dc4a48f76: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } } }
I20260810 00:52:23.260772 23314 tablet_replica.cc:442] T e007fc09469847e0b19485767945f735 P 855f3f1a72704b48a8bea54dc4a48f76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:52:23.308534 23311 raft_consensus.cc:482] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:52:23.308698 23311 raft_consensus.cc:504] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.309980 23311 leader_election.cc:288] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2955df32ef144c688056edf331c6bd99 (127.10.97.66:42311), 855f3f1a72704b48a8bea54dc4a48f76 (127.10.97.67:35309)
I20260810 00:52:23.316759 23233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:23.310444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:42059 (local address 127.10.97.67:35309)
0810 00:52:23.310678 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:23.310685 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:23.311644 (+ 959us) server_negotiation.cc:407] Connection header received
0810 00:52:23.311835 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:23.311840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:23.311901 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:23.311973 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:23.312518 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.313230 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.314851 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.315598 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.315746 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.315838 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:23.316382 (+ 544us) server_negotiation.cc:299] Negotiation successful
0810 00:52:23.316517 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 00:52:23.316785 23316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:23.310214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:42311 (local address 127.10.97.65:60859)
0810 00:52:23.311228 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:23.311245 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:23.311390 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:23.311725 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:23.311730 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:23.311757 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:23.312130 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:23.312144 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.313166 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.313170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:23.314559 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:23.314571 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.315841 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.315849 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:23.316140 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.316175 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 00:52:23.316269 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 00:52:23.316399 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"mutex_wait_us":1,"thread_start_us":780,"threads_started":1}
I20260810 00:52:23.316833 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:23.310308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:60859 (local address 127.10.97.66:42311)
0810 00:52:23.311099 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:23.311106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:23.311330 (+ 224us) server_negotiation.cc:407] Connection header received
0810 00:52:23.311530 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:23.311534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:23.311591 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:23.311681 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:23.312283 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.313028 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.314714 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.315547 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.315707 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.315781 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:23.316350 (+ 569us) server_negotiation.cc:299] Negotiation successful
0810 00:52:23.316547 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":488,"threads_started":1}
I20260810 00:52:23.317346 23320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:23.310371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:35309 (local address 127.10.97.65:42059)
0810 00:52:23.311545 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:23.311560 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:23.311685 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:23.312009 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:23.312013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:23.312037 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:23.312365 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:23.312374 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.313354 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.313358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:23.314707 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:23.314718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.315971 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.315977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:23.316222 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.316250 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 00:52:23.316937 (+ 687us) client_negotiation.cc:240] Negotiation successful
0810 00:52:23.317060 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1047,"mutex_wait_us":65,"thread_start_us":858,"threads_started":1}
I20260810 00:52:23.317585 22935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e007fc09469847e0b19485767945f735" candidate_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2955df32ef144c688056edf331c6bd99" is_pre_election: true
I20260810 00:52:23.317823 22935 raft_consensus.cc:2454] T e007fc09469847e0b19485767945f735 P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e98decf6d77e48ccb5c9c2c62cbced27 in term 0.
I20260810 00:52:23.317909 23134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e007fc09469847e0b19485767945f735" candidate_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "855f3f1a72704b48a8bea54dc4a48f76" is_pre_election: true
I20260810 00:52:23.318176 23134 raft_consensus.cc:2454] T e007fc09469847e0b19485767945f735 P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e98decf6d77e48ccb5c9c2c62cbced27 in term 0.
I20260810 00:52:23.318307 22626 leader_election.cc:302] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2955df32ef144c688056edf331c6bd99, e98decf6d77e48ccb5c9c2c62cbced27; no voters:
I20260810 00:52:23.318691 23311 raft_consensus.cc:2781] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:52:23.318801 23311 raft_consensus.cc:482] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:52:23.318894 23311 raft_consensus.cc:3037] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:23.321148 23311 raft_consensus.cc:504] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.321656 23311 leader_election.cc:288] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 [CANDIDATE]: Term 1 election: Requested vote from peers 2955df32ef144c688056edf331c6bd99 (127.10.97.66:42311), 855f3f1a72704b48a8bea54dc4a48f76 (127.10.97.67:35309)
I20260810 00:52:23.322063 23134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e007fc09469847e0b19485767945f735" candidate_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "855f3f1a72704b48a8bea54dc4a48f76"
I20260810 00:52:23.322079 22935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e007fc09469847e0b19485767945f735" candidate_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2955df32ef144c688056edf331c6bd99"
I20260810 00:52:23.322259 22935 raft_consensus.cc:3037] T e007fc09469847e0b19485767945f735 P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:23.322278 23134 raft_consensus.cc:3037] T e007fc09469847e0b19485767945f735 P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:23.325050 22935 raft_consensus.cc:2454] T e007fc09469847e0b19485767945f735 P 2955df32ef144c688056edf331c6bd99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98decf6d77e48ccb5c9c2c62cbced27 in term 1.
I20260810 00:52:23.325034 23134 raft_consensus.cc:2454] T e007fc09469847e0b19485767945f735 P 855f3f1a72704b48a8bea54dc4a48f76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98decf6d77e48ccb5c9c2c62cbced27 in term 1.
I20260810 00:52:23.325470 22627 leader_election.cc:302] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2955df32ef144c688056edf331c6bd99, 855f3f1a72704b48a8bea54dc4a48f76, e98decf6d77e48ccb5c9c2c62cbced27; no voters:
I20260810 00:52:23.325774 23311 raft_consensus.cc:2781] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:52:23.326140 23311 raft_consensus.cc:686] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 [term 1 LEADER]: Becoming Leader. State: Replica: e98decf6d77e48ccb5c9c2c62cbced27, State: Running, Role: LEADER
I20260810 00:52:23.326478 23311 consensus_queue.cc:237] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.328177 23321 tablet_replica.cc:439] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27: TxnStatusTablet state changed. Reason: New leader e98decf6d77e48ccb5c9c2c62cbced27. Latest consensus state: current_term: 1 leader_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } } }
I20260810 00:52:23.328482 23321 tablet_replica.cc:442] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:52:23.329164 23324 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 00:52:23.329351 23324 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 00:52:23.329711 22491 catalog_manager.cc:5495] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 reported cstate change: term changed from 0 to 1, leader changed from <none> to e98decf6d77e48ccb5c9c2c62cbced27 (127.10.97.65). New cstate: current_term: 1 leader_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } health_report { overall_health: UNKNOWN } } }
I20260810 00:52:23.352013 23290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:23.346595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50090 (local address 127.10.97.65:34977)
0810 00:52:23.346778 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:23.346785 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:23.346953 (+ 168us) server_negotiation.cc:407] Connection header received
0810 00:52:23.347203 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:23.347207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:23.347295 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:23.347413 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:52:23.347756 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.348540 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.349922 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.350340 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.350456 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.350883 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:52:23.351375 (+ 492us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:52:23.351513 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:23.351627 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 00:52:23.351707 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 00:52:23.352048 23303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:23.346655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34977 (local address 127.0.0.1:50090)
0810 00:52:23.346875 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:23.346890 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:23.347025 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:23.347449 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:23.347454 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:23.347466 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:52:23.347626 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:23.347642 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.348690 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.348696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:23.349787 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:23.349800 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.350477 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.350483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:23.350670 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.350723 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:52:23.351480 (+ 757us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:52:23.351512 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 00:52:23.351589 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 00:52:23.351742 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 00:52:23.356105 22935 raft_consensus.cc:1261] T e007fc09469847e0b19485767945f735 P 2955df32ef144c688056edf331c6bd99 [term 1 FOLLOWER]: Refusing update from remote peer e98decf6d77e48ccb5c9c2c62cbced27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:52:23.356122 23134 raft_consensus.cc:1261] T e007fc09469847e0b19485767945f735 P 855f3f1a72704b48a8bea54dc4a48f76 [term 1 FOLLOWER]: Refusing update from remote peer e98decf6d77e48ccb5c9c2c62cbced27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:52:23.356748 23311 consensus_queue.cc:1035] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:52:23.357033 23321 consensus_queue.cc:1035] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:52:23.360595 23252 tablet_replica.cc:439] T e007fc09469847e0b19485767945f735 P 2955df32ef144c688056edf331c6bd99: TxnStatusTablet state changed. Reason: New leader e98decf6d77e48ccb5c9c2c62cbced27. Latest consensus state: current_term: 1 leader_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } } }
I20260810 00:52:23.360894 23252 tablet_replica.cc:442] T e007fc09469847e0b19485767945f735 P 2955df32ef144c688056edf331c6bd99: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:52:23.372253 23252 tablet_replica.cc:439] T e007fc09469847e0b19485767945f735 P 2955df32ef144c688056edf331c6bd99: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } } }
I20260810 00:52:23.372560 23252 tablet_replica.cc:442] T e007fc09469847e0b19485767945f735 P 2955df32ef144c688056edf331c6bd99: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:52:23.374610 23314 tablet_replica.cc:439] T e007fc09469847e0b19485767945f735 P 855f3f1a72704b48a8bea54dc4a48f76: TxnStatusTablet state changed. Reason: New leader e98decf6d77e48ccb5c9c2c62cbced27. Latest consensus state: current_term: 1 leader_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } } }
I20260810 00:52:23.374881 23314 tablet_replica.cc:442] T e007fc09469847e0b19485767945f735 P 855f3f1a72704b48a8bea54dc4a48f76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:52:23.381917 23311 tablet_replica.cc:439] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } } }
I20260810 00:52:23.382951 23311 tablet_replica.cc:442] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:52:23.392766 23321 tablet_replica.cc:439] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } } }
I20260810 00:52:23.393069 23321 tablet_replica.cc:442] T e007fc09469847e0b19485767945f735 P e98decf6d77e48ccb5c9c2c62cbced27: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:52:23.393375 23314 tablet_replica.cc:439] T e007fc09469847e0b19485767945f735 P 855f3f1a72704b48a8bea54dc4a48f76: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } } }
I20260810 00:52:23.393599 23314 tablet_replica.cc:442] T e007fc09469847e0b19485767945f735 P 855f3f1a72704b48a8bea54dc4a48f76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 00:52:23.394594 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:23.385116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53486 (local address 127.10.97.66:42311)
0810 00:52:23.387458 (+ 2342us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:23.387464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:23.387493 (+ 29us) server_negotiation.cc:407] Connection header received
0810 00:52:23.387588 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:23.387593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:23.387702 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:23.387874 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:52:23.388576 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.389651 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.391695 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.392386 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.392484 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.393159 (+ 675us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:52:23.393883 (+ 724us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:52:23.393998 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:23.394249 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 00:52:23.394335 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2215}
I20260810 00:52:23.409387 23290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:23.404240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:46653 (local address 127.10.97.65:34977)
0810 00:52:23.404385 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:23.404390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:23.404917 (+ 527us) server_negotiation.cc:407] Connection header received
0810 00:52:23.405070 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:23.405075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:23.405141 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:23.405223 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:52:23.405668 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.406406 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.407704 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.408192 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.408309 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.408700 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:52:23.408794 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:52:23.408864 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:23.408995 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 00:52:23.409070 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 00:52:23.409449 23348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:23.404275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34977 (local address 127.10.97.66:46653)
0810 00:52:23.404820 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:23.404835 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:23.404961 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:23.405236 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:23.405240 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:23.405251 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:52:23.405516 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:23.405524 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.406530 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.406534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:23.407569 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:23.407580 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.408368 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.408374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:23.408485 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.408536 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:52:23.408883 (+ 347us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:52:23.408895 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 00:52:23.408957 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 00:52:23.409080 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":304,"threads_started":1}
I20260810 00:52:23.418099 23134 raft_consensus.cc:1261] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76 [term 1 FOLLOWER]: Refusing update from remote peer 2955df32ef144c688056edf331c6bd99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:52:23.418217 22733 raft_consensus.cc:1261] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27 [term 1 FOLLOWER]: Refusing update from remote peer 2955df32ef144c688056edf331c6bd99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:52:23.418810 23292 consensus_queue.cc:1035] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:52:23.419437 23252 consensus_queue.cc:1035] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:52:23.424325 23328 mvcc.cc:205] Tried to move back new op lower bound from 7316779595434995712 to 7316779594182254592. Current Snapshot: MvccSnapshot[applied={T|T < 7316779595434995712}]
I20260810 00:52:23.466218 23359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:23.446831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:42311 (local address 127.10.97.65:38475)
0810 00:52:23.460284 (+ 13453us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:23.460305 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:23.460443 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:23.460777 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:23.460782 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:23.460795 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 00:52:23.461192 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:23.461205 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.462613 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.462617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:23.463905 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:23.463920 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.464873 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.464877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:23.465033 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.465103 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 00:52:23.465560 (+ 457us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 00:52:23.465580 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 00:52:23.465664 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 00:52:23.465803 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13257,"thread_start_us":13130,"threads_started":1}
I20260810 00:52:23.466789 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:23.447158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:38475 (local address 127.10.97.66:42311)
0810 00:52:23.447314 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:23.447320 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:23.460389 (+ 13069us) server_negotiation.cc:407] Connection header received
0810 00:52:23.460567 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:23.460572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:23.460660 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:23.460746 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:52:23.461350 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.462497 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.464078 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.464587 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.464712 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.465282 (+ 570us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:52:23.465423 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:52:23.465507 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:23.466443 (+ 936us) server_negotiation.cc:299] Negotiation successful
0810 00:52:23.466521 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 00:52:23.505228 22489 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:52:23.505496 22489 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:52:23.508710 22489 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b: Sending DeleteTablet for 3 replicas of tablet e64ae84fd07d44d09fabdd589af0b2bb
I20260810 00:52:23.509456 22692 tablet_service.cc:1514] Processing DeleteTablet for tablet e64ae84fd07d44d09fabdd589af0b2bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:23 UTC) from {username='oryx', principal='oryx/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:50086
I20260810 00:52:23.509665 22903 tablet_service.cc:1514] Processing DeleteTablet for tablet e64ae84fd07d44d09fabdd589af0b2bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:23 UTC) from {username='oryx', principal='oryx/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:53484
I20260810 00:52:23.510344 23106 tablet_service.cc:1514] Processing DeleteTablet for tablet e64ae84fd07d44d09fabdd589af0b2bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:23 UTC) from {username='oryx', principal='oryx/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:33072
I20260810 00:52:23.521109 23366 tablet_replica.cc:330] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27: stopping tablet replica
I20260810 00:52:23.521422 23366 raft_consensus.cc:2229] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:52:23.521739 23366 raft_consensus.cc:2258] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:23.522745 23366 ts_tablet_manager.cc:1905] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:23.529920 23366 ts_tablet_manager.cc:1918] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:52:23.530115 23366 log.cc:1196] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/e64ae84fd07d44d09fabdd589af0b2bb
I20260810 00:52:23.530632 23366 ts_tablet_manager.cc:1939] T e64ae84fd07d44d09fabdd589af0b2bb P e98decf6d77e48ccb5c9c2c62cbced27: Deleting consensus metadata
I20260810 00:52:23.531718 22472 catalog_manager.cc:4841] TS e98decf6d77e48ccb5c9c2c62cbced27 (127.10.97.65:34977): tablet e64ae84fd07d44d09fabdd589af0b2bb (table test-table [id=b3f17bebca0e43c0bd79bcefb6f1e88e]) successfully deleted
I20260810 00:52:23.535938 23368 tablet_replica.cc:330] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76: stopping tablet replica
I20260810 00:52:23.536144 23368 raft_consensus.cc:2229] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:52:23.536377 23367 tablet_replica.cc:330] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99: stopping tablet replica
I20260810 00:52:23.536402 23368 raft_consensus.cc:2258] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:23.536649 23367 raft_consensus.cc:2229] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:52:23.537307 23367 raft_consensus.cc:2258] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:23.537560 23368 ts_tablet_manager.cc:1905] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:23.539961 23367 ts_tablet_manager.cc:1905] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:23.544392 23368 ts_tablet_manager.cc:1918] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:52:23.544520 23368 log.cc:1196] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/e64ae84fd07d44d09fabdd589af0b2bb
I20260810 00:52:23.544975 23368 ts_tablet_manager.cc:1939] T e64ae84fd07d44d09fabdd589af0b2bb P 855f3f1a72704b48a8bea54dc4a48f76: Deleting consensus metadata
I20260810 00:52:23.545994 22472 catalog_manager.cc:4841] TS 855f3f1a72704b48a8bea54dc4a48f76 (127.10.97.67:35309): tablet e64ae84fd07d44d09fabdd589af0b2bb (table test-table [id=b3f17bebca0e43c0bd79bcefb6f1e88e]) successfully deleted
I20260810 00:52:23.546414 23367 ts_tablet_manager.cc:1918] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:52:23.546588 23367 log.cc:1196] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/e64ae84fd07d44d09fabdd589af0b2bb
I20260810 00:52:23.547062 23367 ts_tablet_manager.cc:1939] T e64ae84fd07d44d09fabdd589af0b2bb P 2955df32ef144c688056edf331c6bd99: Deleting consensus metadata
I20260810 00:52:23.548197 22471 catalog_manager.cc:4841] TS 2955df32ef144c688056edf331c6bd99 (127.10.97.66:42311): tablet e64ae84fd07d44d09fabdd589af0b2bb (table test-table [id=b3f17bebca0e43c0bd79bcefb6f1e88e]) successfully deleted
Aug 10 00:52:23 dist-test-slave-jlzn krb5kdc[22426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 00:52:23 dist-test-slave-jlzn krb5kdc[22426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:23 dist-test-slave-jlzn krb5kdc[22426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 00:52:23.588096 23302 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:23.577434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48060 (local address 127.10.97.126:42341)
0810 00:52:23.577658 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:23.577666 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:23.578184 (+ 518us) server_negotiation.cc:407] Connection header received
0810 00:52:23.578377 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:23.578382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:23.578450 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:23.578533 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:23.579934 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.580681 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.581931 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.582475 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.582597 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.587842 (+ 5245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48060: BlockingRecv error: recv got EOF from 127.0.0.1:48060 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 10 00:52:23 dist-test-slave-jlzn krb5kdc[22426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM
I20260810 00:52:23.604533 23302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:23.591103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48068 (local address 127.10.97.126:42341)
0810 00:52:23.591256 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:23.591261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:23.591278 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:52:23.591460 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:23.591465 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:23.591536 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:23.591588 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:23.592936 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.593814 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.594995 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.595417 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.595522 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.599019 (+ 3497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:23.599024 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:23.599028 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:23.599069 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:23.601796 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:23.602572 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:23.602579 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:23.602583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:23.602725 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:23.603173 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:23.603179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:23.603182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:23.603434 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:23.603694 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:23.603975 (+ 281us) server_negotiation.cc:299] Negotiation successful
0810 00:52:23.604180 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 00:52:23.607661 22489 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:52:23.608299 22489 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:52:23.614413 22692 tablet_service.cc:1467] Processing CreateTablet for tablet 1bd1be0c168e462891ceba732a3b9d26 (DEFAULT_TABLE table=test-table [id=2874c654bed04383bc8c16e3be3eef3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:23.614861 22692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bd1be0c168e462891ceba732a3b9d26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:23.619601 23241 tablet_bootstrap.cc:492] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27: Bootstrap starting.
I20260810 00:52:23.620555 23106 tablet_service.cc:1467] Processing CreateTablet for tablet 1bd1be0c168e462891ceba732a3b9d26 (DEFAULT_TABLE table=test-table [id=2874c654bed04383bc8c16e3be3eef3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:23.620981 23106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bd1be0c168e462891ceba732a3b9d26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:23.621377 23241 tablet_bootstrap.cc:654] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:23.628396 22903 tablet_service.cc:1467] Processing CreateTablet for tablet 1bd1be0c168e462891ceba732a3b9d26 (DEFAULT_TABLE table=test-table [id=2874c654bed04383bc8c16e3be3eef3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:23.628858 22903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bd1be0c168e462891ceba732a3b9d26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:23.631472 23241 tablet_bootstrap.cc:492] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27: No bootstrap required, opened a new log
I20260810 00:52:23.631698 23241 ts_tablet_manager.cc:1397] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.006s
I20260810 00:52:23.633002 23241 raft_consensus.cc:348] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.633378 23241 raft_consensus.cc:374] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:23.633520 23241 raft_consensus.cc:729] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e98decf6d77e48ccb5c9c2c62cbced27, State: Initialized, Role: FOLLOWER
I20260810 00:52:23.633826 23241 consensus_queue.cc:260] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.635159 23241 ts_tablet_manager.cc:1428] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 00:52:23.638310 23239 tablet_bootstrap.cc:492] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99: Bootstrap starting.
I20260810 00:52:23.638821 23306 tablet_bootstrap.cc:492] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76: Bootstrap starting.
I20260810 00:52:23.639945 23239 tablet_bootstrap.cc:654] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:23.640437 23306 tablet_bootstrap.cc:654] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:23.650833 23306 tablet_bootstrap.cc:492] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76: No bootstrap required, opened a new log
I20260810 00:52:23.651011 23306 ts_tablet_manager.cc:1397] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260810 00:52:23.651754 23306 raft_consensus.cc:348] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.652143 23306 raft_consensus.cc:374] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:23.652261 23306 raft_consensus.cc:729] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 855f3f1a72704b48a8bea54dc4a48f76, State: Initialized, Role: FOLLOWER
I20260810 00:52:23.652566 23306 consensus_queue.cc:260] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.653209 23306 ts_tablet_manager.cc:1428] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 00:52:23.654090 23239 tablet_bootstrap.cc:492] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99: No bootstrap required, opened a new log
I20260810 00:52:23.654254 23239 ts_tablet_manager.cc:1397] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.004s
I20260810 00:52:23.655047 23239 raft_consensus.cc:348] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.655304 23239 raft_consensus.cc:374] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:23.655396 23239 raft_consensus.cc:729] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2955df32ef144c688056edf331c6bd99, State: Initialized, Role: FOLLOWER
I20260810 00:52:23.655633 23239 consensus_queue.cc:260] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.656198 23239 ts_tablet_manager.cc:1428] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 00:52:23.672011 23252 raft_consensus.cc:482] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:52:23.672194 23252 raft_consensus.cc:504] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.672912 23252 leader_election.cc:288] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e98decf6d77e48ccb5c9c2c62cbced27 (127.10.97.65:34977), 855f3f1a72704b48a8bea54dc4a48f76 (127.10.97.67:35309)
I20260810 00:52:23.674047 22733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bd1be0c168e462891ceba732a3b9d26" candidate_uuid: "2955df32ef144c688056edf331c6bd99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" is_pre_election: true
I20260810 00:52:23.674304 22733 raft_consensus.cc:2454] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2955df32ef144c688056edf331c6bd99 in term 0.
I20260810 00:52:23.674950 23133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bd1be0c168e462891ceba732a3b9d26" candidate_uuid: "2955df32ef144c688056edf331c6bd99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "855f3f1a72704b48a8bea54dc4a48f76" is_pre_election: true
I20260810 00:52:23.675259 23133 raft_consensus.cc:2454] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2955df32ef144c688056edf331c6bd99 in term 0.
I20260810 00:52:23.676044 22830 leader_election.cc:302] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2955df32ef144c688056edf331c6bd99, e98decf6d77e48ccb5c9c2c62cbced27; no voters:
I20260810 00:52:23.676563 23252 raft_consensus.cc:2781] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:52:23.676697 23252 raft_consensus.cc:482] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:52:23.676800 23252 raft_consensus.cc:3037] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:23.679942 23252 raft_consensus.cc:504] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.680629 23252 leader_election.cc:288] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [CANDIDATE]: Term 1 election: Requested vote from peers e98decf6d77e48ccb5c9c2c62cbced27 (127.10.97.65:34977), 855f3f1a72704b48a8bea54dc4a48f76 (127.10.97.67:35309)
I20260810 00:52:23.681130 23133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bd1be0c168e462891ceba732a3b9d26" candidate_uuid: "2955df32ef144c688056edf331c6bd99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "855f3f1a72704b48a8bea54dc4a48f76"
I20260810 00:52:23.681233 22733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bd1be0c168e462891ceba732a3b9d26" candidate_uuid: "2955df32ef144c688056edf331c6bd99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98decf6d77e48ccb5c9c2c62cbced27"
I20260810 00:52:23.681339 23133 raft_consensus.cc:3037] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:23.681460 22733 raft_consensus.cc:3037] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:23.684297 23133 raft_consensus.cc:2454] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2955df32ef144c688056edf331c6bd99 in term 1.
I20260810 00:52:23.684751 22733 raft_consensus.cc:2454] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2955df32ef144c688056edf331c6bd99 in term 1.
I20260810 00:52:23.684788 22830 leader_election.cc:302] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2955df32ef144c688056edf331c6bd99, 855f3f1a72704b48a8bea54dc4a48f76; no voters:
I20260810 00:52:23.685470 23252 raft_consensus.cc:2781] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:52:23.685663 23252 raft_consensus.cc:686] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [term 1 LEADER]: Becoming Leader. State: Replica: 2955df32ef144c688056edf331c6bd99, State: Running, Role: LEADER
I20260810 00:52:23.686283 23252 consensus_queue.cc:237] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.689262 22489 catalog_manager.cc:5495] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2955df32ef144c688056edf331c6bd99 (127.10.97.66). New cstate: current_term: 1 leader_uuid: "2955df32ef144c688056edf331c6bd99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } health_report { overall_health: UNKNOWN } } }
I20260810 00:52:23.711230 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:23.704412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53490 (local address 127.10.97.66:42311)
0810 00:52:23.704585 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:23.704592 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:23.704614 (+ 22us) server_negotiation.cc:407] Connection header received
0810 00:52:23.704694 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:23.704700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:23.704784 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:23.704874 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:52:23.705482 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.706599 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.708632 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:23.709348 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:23.709465 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:23.710052 (+ 587us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:52:23.710211 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:52:23.710294 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:23.710849 (+ 555us) server_negotiation.cc:299] Negotiation successful
0810 00:52:23.710933 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 00:52:23.714205 23133 raft_consensus.cc:1261] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76 [term 1 FOLLOWER]: Refusing update from remote peer 2955df32ef144c688056edf331c6bd99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:52:23.714593 22733 raft_consensus.cc:1261] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27 [term 1 FOLLOWER]: Refusing update from remote peer 2955df32ef144c688056edf331c6bd99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:52:23.715046 23292 consensus_queue.cc:1035] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:52:23.715286 23252 consensus_queue.cc:1035] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:52:23.746035 22488 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:52:23.746380 22488 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:52:23.751258 22488 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b: Sending DeleteTablet for 3 replicas of tablet 1bd1be0c168e462891ceba732a3b9d26
I20260810 00:52:23.752904 22692 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bd1be0c168e462891ceba732a3b9d26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:23 UTC) from {username='oryx', principal='oryx/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:50086
I20260810 00:52:23.753206 23366 tablet_replica.cc:330] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27: stopping tablet replica
I20260810 00:52:23.753794 23106 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bd1be0c168e462891ceba732a3b9d26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:23 UTC) from {username='oryx', principal='oryx/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:33072
I20260810 00:52:23.754107 23368 tablet_replica.cc:330] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76: stopping tablet replica
I20260810 00:52:23.754302 23368 raft_consensus.cc:2229] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:52:23.754334 22903 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bd1be0c168e462891ceba732a3b9d26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:23 UTC) from {username='oryx', principal='oryx/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:53484
I20260810 00:52:23.754565 23368 raft_consensus.cc:2258] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:23.754612 23367 tablet_replica.cc:330] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99: stopping tablet replica
I20260810 00:52:23.754802 23367 raft_consensus.cc:2229] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:52:23.755117 23367 raft_consensus.cc:2258] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:23.755852 22488 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 00:52:23.756129 23368 ts_tablet_manager.cc:1905] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 00:52:23.756541 22488 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:52:23.756909 23367 ts_tablet_manager.cc:1905] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:23.759917 23366 raft_consensus.cc:2229] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:52:23.760573 23366 raft_consensus.cc:2258] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:23.762380 23368 ts_tablet_manager.cc:1918] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:52:23.762547 23368 log.cc:1196] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/1bd1be0c168e462891ceba732a3b9d26
I20260810 00:52:23.762984 23368 ts_tablet_manager.cc:1939] T 1bd1be0c168e462891ceba732a3b9d26 P 855f3f1a72704b48a8bea54dc4a48f76: Deleting consensus metadata
I20260810 00:52:23.764104 22472 catalog_manager.cc:4841] TS 855f3f1a72704b48a8bea54dc4a48f76 (127.10.97.67:35309): tablet 1bd1be0c168e462891ceba732a3b9d26 (table test-table [id=2874c654bed04383bc8c16e3be3eef3b]) successfully deleted
I20260810 00:52:23.768405 23367 ts_tablet_manager.cc:1918] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:52:23.768481 23106 tablet_service.cc:1467] Processing CreateTablet for tablet c6680f8e3e5741909ea16f3ec73c12dc (DEFAULT_TABLE table=test-table [id=d104fe6609f94e18b715bbd23c638017]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:23.768920 23106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6680f8e3e5741909ea16f3ec73c12dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:23.769654 23366 ts_tablet_manager.cc:1905] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:23.772145 22692 tablet_service.cc:1467] Processing CreateTablet for tablet c6680f8e3e5741909ea16f3ec73c12dc (DEFAULT_TABLE table=test-table [id=d104fe6609f94e18b715bbd23c638017]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:23.772538 22692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6680f8e3e5741909ea16f3ec73c12dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:23.776312 23367 log.cc:1196] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/1bd1be0c168e462891ceba732a3b9d26
I20260810 00:52:23.776437 22903 tablet_service.cc:1467] Processing CreateTablet for tablet c6680f8e3e5741909ea16f3ec73c12dc (DEFAULT_TABLE table=test-table [id=d104fe6609f94e18b715bbd23c638017]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:23.776895 22903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6680f8e3e5741909ea16f3ec73c12dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:23.776901 23367 ts_tablet_manager.cc:1939] T 1bd1be0c168e462891ceba732a3b9d26 P 2955df32ef144c688056edf331c6bd99: Deleting consensus metadata
I20260810 00:52:23.781788 22471 catalog_manager.cc:4841] TS 2955df32ef144c688056edf331c6bd99 (127.10.97.66:42311): tablet 1bd1be0c168e462891ceba732a3b9d26 (table test-table [id=2874c654bed04383bc8c16e3be3eef3b]) successfully deleted
I20260810 00:52:23.782410 23306 tablet_bootstrap.cc:492] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76: Bootstrap starting.
I20260810 00:52:23.783728 23239 tablet_bootstrap.cc:492] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99: Bootstrap starting.
I20260810 00:52:23.784180 23306 tablet_bootstrap.cc:654] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:23.785262 23239 tablet_bootstrap.cc:654] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:23.793376 23241 tablet_bootstrap.cc:492] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27: Bootstrap starting.
I20260810 00:52:23.795122 23241 tablet_bootstrap.cc:654] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:23.793385 23366 ts_tablet_manager.cc:1918] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:52:23.796378 23366 log.cc:1196] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/1bd1be0c168e462891ceba732a3b9d26
I20260810 00:52:23.796893 23366 ts_tablet_manager.cc:1939] T 1bd1be0c168e462891ceba732a3b9d26 P e98decf6d77e48ccb5c9c2c62cbced27: Deleting consensus metadata
I20260810 00:52:23.798806 22472 catalog_manager.cc:4841] TS e98decf6d77e48ccb5c9c2c62cbced27 (127.10.97.65:34977): tablet 1bd1be0c168e462891ceba732a3b9d26 (table test-table [id=2874c654bed04383bc8c16e3be3eef3b]) successfully deleted
I20260810 00:52:23.804203 23306 tablet_bootstrap.cc:492] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76: No bootstrap required, opened a new log
I20260810 00:52:23.804416 23306 ts_tablet_manager.cc:1397] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76: Time spent bootstrapping tablet: real 0.022s user 0.005s sys 0.003s
I20260810 00:52:23.805274 23306 raft_consensus.cc:348] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.805631 23306 raft_consensus.cc:374] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:23.805783 23306 raft_consensus.cc:729] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 855f3f1a72704b48a8bea54dc4a48f76, State: Initialized, Role: FOLLOWER
I20260810 00:52:23.806094 23306 consensus_queue.cc:260] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.807277 23239 tablet_bootstrap.cc:492] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99: No bootstrap required, opened a new log
I20260810 00:52:23.807464 23239 ts_tablet_manager.cc:1397] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.007s
I20260810 00:52:23.808279 23239 raft_consensus.cc:348] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.808537 23239 raft_consensus.cc:374] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:23.808620 23239 raft_consensus.cc:729] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2955df32ef144c688056edf331c6bd99, State: Initialized, Role: FOLLOWER
I20260810 00:52:23.808904 23239 consensus_queue.cc:260] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.809397 23239 ts_tablet_manager.cc:1428] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 00:52:23.810205 23306 ts_tablet_manager.cc:1428] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260810 00:52:23.817502 23241 tablet_bootstrap.cc:492] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27: No bootstrap required, opened a new log
I20260810 00:52:23.819675 23241 ts_tablet_manager.cc:1397] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.003s
I20260810 00:52:23.820539 23241 raft_consensus.cc:348] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.820782 23241 raft_consensus.cc:374] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:23.820866 23241 raft_consensus.cc:729] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e98decf6d77e48ccb5c9c2c62cbced27, State: Initialized, Role: FOLLOWER
I20260810 00:52:23.821115 23241 consensus_queue.cc:260] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.822539 23241 ts_tablet_manager.cc:1428] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 00:52:23.976619 23252 raft_consensus.cc:482] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:52:23.976836 23252 raft_consensus.cc:504] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.977504 23252 leader_election.cc:288] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e98decf6d77e48ccb5c9c2c62cbced27 (127.10.97.65:34977), 855f3f1a72704b48a8bea54dc4a48f76 (127.10.97.67:35309)
I20260810 00:52:23.978080 22733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6680f8e3e5741909ea16f3ec73c12dc" candidate_uuid: "2955df32ef144c688056edf331c6bd99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" is_pre_election: true
I20260810 00:52:23.978303 22733 raft_consensus.cc:2454] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2955df32ef144c688056edf331c6bd99 in term 0.
I20260810 00:52:23.978217 23134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6680f8e3e5741909ea16f3ec73c12dc" candidate_uuid: "2955df32ef144c688056edf331c6bd99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "855f3f1a72704b48a8bea54dc4a48f76" is_pre_election: true
I20260810 00:52:23.978492 23134 raft_consensus.cc:2454] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2955df32ef144c688056edf331c6bd99 in term 0.
I20260810 00:52:23.978731 22830 leader_election.cc:302] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2955df32ef144c688056edf331c6bd99, e98decf6d77e48ccb5c9c2c62cbced27; no voters:
I20260810 00:52:23.979084 23252 raft_consensus.cc:2781] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:52:23.979216 23252 raft_consensus.cc:482] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:52:23.979308 23252 raft_consensus.cc:3037] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:23.981514 23252 raft_consensus.cc:504] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.982048 23252 leader_election.cc:288] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [CANDIDATE]: Term 1 election: Requested vote from peers e98decf6d77e48ccb5c9c2c62cbced27 (127.10.97.65:34977), 855f3f1a72704b48a8bea54dc4a48f76 (127.10.97.67:35309)
I20260810 00:52:23.982450 22733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6680f8e3e5741909ea16f3ec73c12dc" candidate_uuid: "2955df32ef144c688056edf331c6bd99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98decf6d77e48ccb5c9c2c62cbced27"
I20260810 00:52:23.982546 23134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6680f8e3e5741909ea16f3ec73c12dc" candidate_uuid: "2955df32ef144c688056edf331c6bd99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "855f3f1a72704b48a8bea54dc4a48f76"
I20260810 00:52:23.982689 22733 raft_consensus.cc:3037] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:23.982733 23134 raft_consensus.cc:3037] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:23.984889 23134 raft_consensus.cc:2454] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2955df32ef144c688056edf331c6bd99 in term 1.
I20260810 00:52:23.985306 22830 leader_election.cc:302] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2955df32ef144c688056edf331c6bd99, 855f3f1a72704b48a8bea54dc4a48f76; no voters:
I20260810 00:52:23.985618 23252 raft_consensus.cc:2781] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:52:23.985692 22733 raft_consensus.cc:2454] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2955df32ef144c688056edf331c6bd99 in term 1.
I20260810 00:52:23.986074 23252 raft_consensus.cc:686] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [term 1 LEADER]: Becoming Leader. State: Replica: 2955df32ef144c688056edf331c6bd99, State: Running, Role: LEADER
I20260810 00:52:23.986385 23252 consensus_queue.cc:237] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } }
I20260810 00:52:23.989207 22488 catalog_manager.cc:5495] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2955df32ef144c688056edf331c6bd99 (127.10.97.66). New cstate: current_term: 1 leader_uuid: "2955df32ef144c688056edf331c6bd99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2955df32ef144c688056edf331c6bd99" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 } health_report { overall_health: UNKNOWN } } }
I20260810 00:52:24.012926 22733 raft_consensus.cc:1261] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27 [term 1 FOLLOWER]: Refusing update from remote peer 2955df32ef144c688056edf331c6bd99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:52:24.012948 23134 raft_consensus.cc:1261] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76 [term 1 FOLLOWER]: Refusing update from remote peer 2955df32ef144c688056edf331c6bd99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:52:24.013507 23292 consensus_queue.cc:1035] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e98decf6d77e48ccb5c9c2c62cbced27" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:52:24.013814 23252 consensus_queue.cc:1035] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "855f3f1a72704b48a8bea54dc4a48f76" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 35309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:52:24.062629 22489 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:52:24.062899 22489 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:52:24.065533 22489 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cbde9124c7bb4285994cfc3e1314a80b: Sending DeleteTablet for 3 replicas of tablet c6680f8e3e5741909ea16f3ec73c12dc
I20260810 00:52:24.066323 22903 tablet_service.cc:1514] Processing DeleteTablet for tablet c6680f8e3e5741909ea16f3ec73c12dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:24 UTC) from {username='oryx', principal='oryx/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:53484
I20260810 00:52:24.066381 23106 tablet_service.cc:1514] Processing DeleteTablet for tablet c6680f8e3e5741909ea16f3ec73c12dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:24 UTC) from {username='oryx', principal='oryx/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:33072
I20260810 00:52:24.066435 22692 tablet_service.cc:1514] Processing DeleteTablet for tablet c6680f8e3e5741909ea16f3ec73c12dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:24 UTC) from {username='oryx', principal='oryx/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:50086
I20260810 00:52:24.066650 23367 tablet_replica.cc:330] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99: stopping tablet replica
I20260810 00:52:24.066649 23368 tablet_replica.cc:330] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76: stopping tablet replica
I20260810 00:52:24.066692 23366 tablet_replica.cc:330] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27: stopping tablet replica
I20260810 00:52:24.066817 23366 raft_consensus.cc:2229] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:52:24.066818 23368 raft_consensus.cc:2229] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:52:24.066817 23367 raft_consensus.cc:2229] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:52:24.066951 23366 raft_consensus.cc:2258] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:24.066998 23368 raft_consensus.cc:2258] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:24.067113 23367 raft_consensus.cc:2258] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:24.068027 23367 ts_tablet_manager.cc:1905] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:24.068037 23366 ts_tablet_manager.cc:1905] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:24.068357 23368 ts_tablet_manager.cc:1905] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:24.068413 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 22596
I20260810 00:52:24.074306 23367 ts_tablet_manager.cc:1918] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:52:24.074306 23366 ts_tablet_manager.cc:1918] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:52:24.074436 23366 log.cc:1196] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/c6680f8e3e5741909ea16f3ec73c12dc
I20260810 00:52:24.074436 23367 log.cc:1196] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/c6680f8e3e5741909ea16f3ec73c12dc
I20260810 00:52:24.074851 23366 ts_tablet_manager.cc:1939] T c6680f8e3e5741909ea16f3ec73c12dc P e98decf6d77e48ccb5c9c2c62cbced27: Deleting consensus metadata
I20260810 00:52:24.074851 23367 ts_tablet_manager.cc:1939] T c6680f8e3e5741909ea16f3ec73c12dc P 2955df32ef144c688056edf331c6bd99: Deleting consensus metadata
I20260810 00:52:24.075309 23368 ts_tablet_manager.cc:1918] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 00:52:24.075435 23368 log.cc:1196] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/c6680f8e3e5741909ea16f3ec73c12dc
I20260810 00:52:24.075894 23368 ts_tablet_manager.cc:1939] T c6680f8e3e5741909ea16f3ec73c12dc P 855f3f1a72704b48a8bea54dc4a48f76: Deleting consensus metadata
I20260810 00:52:24.076028 22472 catalog_manager.cc:4841] TS e98decf6d77e48ccb5c9c2c62cbced27 (127.10.97.65:34977): tablet c6680f8e3e5741909ea16f3ec73c12dc (table test-table [id=d104fe6609f94e18b715bbd23c638017]) successfully deleted
I20260810 00:52:24.076308 22471 catalog_manager.cc:4841] TS 2955df32ef144c688056edf331c6bd99 (127.10.97.66:42311): tablet c6680f8e3e5741909ea16f3ec73c12dc (table test-table [id=d104fe6609f94e18b715bbd23c638017]) successfully deleted
I20260810 00:52:24.076843 22472 catalog_manager.cc:4841] TS 855f3f1a72704b48a8bea54dc4a48f76 (127.10.97.67:35309): tablet c6680f8e3e5741909ea16f3ec73c12dc (table test-table [id=d104fe6609f94e18b715bbd23c638017]) successfully deleted
Aug 10 00:52:24 dist-test-slave-jlzn krb5kdc[22426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.97.65@KRBTEST.COM
I20260810 00:52:24.089411 23411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:24.075768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50104 (local address 127.10.97.65:34977)
0810 00:52:24.076929 (+ 1161us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:24.076935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:24.076952 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:52:24.077001 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:24.077006 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:24.077099 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:24.077185 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:24.078312 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:24.078999 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:24.080210 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:24.080587 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:24.080677 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:24.084413 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:24.084426 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:24.084442 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:24.084470 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:24.087164 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:24.087747 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:24.087754 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:24.087756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:24.087953 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:24.088247 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:24.088250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:24.088253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:24.088552 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:24.088799 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:24.088951 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 00:52:24.089104 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1029,"thread_start_us":911,"threads_started":1}
I20260810 00:52:24.090006 22792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:24.221453 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 22596
I20260810 00:52:24.250891 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 22802
Aug 10 00:52:24 dist-test-slave-jlzn krb5kdc[22426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.97.66@KRBTEST.COM
I20260810 00:52:24.264755 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:24.251601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53500 (local address 127.10.97.66:42311)
0810 00:52:24.252037 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:24.252041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:24.252058 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:52:24.252122 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:24.252126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:24.252199 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:24.252291 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:24.253433 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:24.254120 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:24.255399 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:24.255867 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:24.255965 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:24.259658 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:24.259671 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:24.259686 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:24.259717 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:24.262369 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:24.262953 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:24.262962 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:24.262966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:24.263130 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:24.263439 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:24.263442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:24.263444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:24.263776 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:24.264060 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:24.264338 (+ 278us) server_negotiation.cc:299] Negotiation successful
0810 00:52:24.264501 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":242,"threads_started":1}
I20260810 00:52:24.265344 22997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:24.385699 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 22802
I20260810 00:52:24.413159 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 23010
Aug 10 00:52:24 dist-test-slave-jlzn krb5kdc[22426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.97.67@KRBTEST.COM
I20260810 00:52:24.427642 23418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:24.413860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33096 (local address 127.10.97.67:35309)
0810 00:52:24.414325 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:24.414330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:24.414344 (+ 14us) server_negotiation.cc:407] Connection header received
0810 00:52:24.414405 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:24.414409 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:24.414466 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:24.414566 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:24.415857 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:24.416571 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:24.417834 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:24.418221 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:24.418330 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:24.422609 (+ 4279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:24.422623 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:24.422640 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:24.422673 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:24.425107 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:24.425698 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:24.425705 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:24.425707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:24.425849 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:24.426197 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:24.426201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:24.426203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:24.426545 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:24.426844 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:24.427121 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 00:52:24.427346 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":220,"threads_started":1}
I20260810 00:52:24.428299 23194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:24.545907 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 23010
I20260810 00:52:24.575652 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 22446
I20260810 00:52:24.576809 22550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:24.690184 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 22446
2026-08-10T00:52:24Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6862 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:52:24 dist-test-slave-jlzn krb5kdc[23426](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:52:24 dist-test-slave-jlzn krb5kdc[23426](info): set up 2 sockets
Aug 10 00:52:24 dist-test-slave-jlzn krb5kdc[23426](info): commencing operation
krb5kdc: starting...
W20260810 00:52:25.876946 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.003s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:52:26 dist-test-slave-jlzn krb5kdc[23426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:52:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:52:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:52:26.090865 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:38697
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:36761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:38697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:26.400571 23445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:26.400839 23445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:26.400970 23445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:26.407900 23445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:52:26.408000 23445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:26.408074 23445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:26.408150 23445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:52:26.408215 23445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:52:26.417544 23445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:36761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:38697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:38697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:26.418676 23445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:26.419926 23445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:26.430859 23452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:26.430811 23456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:26.430892 23453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:26.431663 23445 server_base.cc:1034] running on GCE node
I20260810 00:52:26.432420 23445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:26.433691 23445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:26.434890 23445 hybrid_clock.cc:648] HybridClock initialized: now 1786323146434838 us; error 40 us; skew 500 ppm
I20260810 00:52:26.435197 23445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T00:52:26Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1809 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:52:26 dist-test-slave-jlzn krb5kdc[23465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:52:26 dist-test-slave-jlzn krb5kdc[23465](info): set up 2 sockets
Aug 10 00:52:26 dist-test-slave-jlzn krb5kdc[23465](info): commencing operation
krb5kdc: starting...
W20260810 00:52:27.692761 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.001s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:52:27 dist-test-slave-jlzn krb5kdc[23465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:52:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:52:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:52:27.927004 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:33165
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:33165
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:28.237973 23483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:28.238241 23483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:28.238368 23483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:28.245180 23483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:52:28.245276 23483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:28.245352 23483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:28.245422 23483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:52:28.245481 23483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:52:28.254709 23483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:33165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:33165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:28.255875 23483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:28.257076 23483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:28.267843 23489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:28.267782 23490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:28.267879 23493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:28.268801 23483 server_base.cc:1034] running on GCE node
I20260810 00:52:28.269574 23483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:28.270850 23483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:28.272043 23483 hybrid_clock.cc:648] HybridClock initialized: now 1786323148271998 us; error 29 us; skew 500 ppm
I20260810 00:52:28.272343 23483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:28 dist-test-slave-jlzn krb5kdc[23465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323148, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:28.275226 23483 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:52:28.276592 23483 webserver.cc:460] Webserver started at http://127.10.97.126:45083/ using document root <none> and password file <none>
I20260810 00:52:28.277348 23483 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:28.277477 23483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:28.277807 23483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:28.280633 23483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "92b3cdd88e3342f9a66bef302a68d7e3"
format_stamp: "Formatted at 2026-08-10 00:52:28 on dist-test-slave-jlzn"
server_key: "750efbed8b8106e79cb65bdbb8b2684a"
server_key_iv: "0e67be58d797322fdc9a063656f70a22"
server_key_version: "encryptionkey@0"
I20260810 00:52:28.281347 23483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "92b3cdd88e3342f9a66bef302a68d7e3"
format_stamp: "Formatted at 2026-08-10 00:52:28 on dist-test-slave-jlzn"
server_key: "750efbed8b8106e79cb65bdbb8b2684a"
server_key_iv: "0e67be58d797322fdc9a063656f70a22"
server_key_version: "encryptionkey@0"
I20260810 00:52:28.286032 23483 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260810 00:52:28.289175 23500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:28.290303 23483 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:52:28.290459 23483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "92b3cdd88e3342f9a66bef302a68d7e3"
format_stamp: "Formatted at 2026-08-10 00:52:28 on dist-test-slave-jlzn"
server_key: "750efbed8b8106e79cb65bdbb8b2684a"
server_key_iv: "0e67be58d797322fdc9a063656f70a22"
server_key_version: "encryptionkey@0"
I20260810 00:52:28.290627 23483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:28.307669 23483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:28.311471 23483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:28.311733 23483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:28.328200 23483 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:33165
I20260810 00:52:28.328235 23581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:33165 every 8 connection(s)
I20260810 00:52:28.329430 23483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:52:28.332965 23585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:28.334131 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 23483
I20260810 00:52:28.334375 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:52:28.335188 10629 external_mini_cluster.cc:1428] Setting key 5f24d1c7a1ab2ccdb69c71f192984260
I20260810 00:52:28.346597 23585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3: Bootstrap starting.
Aug 10 00:52:28 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:52:28.350193 23585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:28.351260 23585 log.cc:824] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:28.354209 23585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3: No bootstrap required, opened a new log
I20260810 00:52:28.357312 23589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:28.337131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56012 (local address 127.10.97.126:33165)
0810 00:52:28.337865 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:28.337878 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:28.337924 (+ 46us) server_negotiation.cc:407] Connection header received
0810 00:52:28.339484 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:28.339504 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:28.339890 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:28.340210 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:28.341907 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:28.343175 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:28.344941 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:28.345569 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:28.345690 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:28.350206 (+ 4516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:28.350216 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:28.350229 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:28.350273 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:28.353379 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:28.354123 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:28.354128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:28.354133 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:28.354322 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:28.354679 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:28.354682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:28.354685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:28.355193 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:28.355464 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:28.355982 (+ 518us) server_negotiation.cc:299] Negotiation successful
0810 00:52:28.356362 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":335,"threads_started":1}
I20260810 00:52:28.361692 23585 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b3cdd88e3342f9a66bef302a68d7e3" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33165 } }
I20260810 00:52:28.362001 23585 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:28.362090 23585 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92b3cdd88e3342f9a66bef302a68d7e3, State: Initialized, Role: FOLLOWER
I20260810 00:52:28.362591 23585 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b3cdd88e3342f9a66bef302a68d7e3" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33165 } }
I20260810 00:52:28.362800 23585 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:52:28.362929 23585 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:52:28.363082 23585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:28.365191 23585 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b3cdd88e3342f9a66bef302a68d7e3" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33165 } }
I20260810 00:52:28.365711 23585 leader_election.cc:302] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92b3cdd88e3342f9a66bef302a68d7e3; no voters:
I20260810 00:52:28.366276 23585 leader_election.cc:288] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:52:28.366348 23591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:52:28.367015 23591 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [term 1 LEADER]: Becoming Leader. State: Replica: 92b3cdd88e3342f9a66bef302a68d7e3, State: Running, Role: LEADER
I20260810 00:52:28.367496 23591 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b3cdd88e3342f9a66bef302a68d7e3" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33165 } }
I20260810 00:52:28.368242 23585 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:52:28.370296 23593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92b3cdd88e3342f9a66bef302a68d7e3. Latest consensus state: current_term: 1 leader_uuid: "92b3cdd88e3342f9a66bef302a68d7e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b3cdd88e3342f9a66bef302a68d7e3" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33165 } } }
I20260810 00:52:28.370464 23592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92b3cdd88e3342f9a66bef302a68d7e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b3cdd88e3342f9a66bef302a68d7e3" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33165 } } }
I20260810 00:52:28.370766 23592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:28.370771 23593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:28.373245 23598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:52:28.377313 23598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:52:28.385030 23598 catalog_manager.cc:1347] Generated new cluster ID: bc8587967ecb45a882b68a080563b1da
I20260810 00:52:28.385133 23598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:52:28.400081 23598 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:52:28.401752 23598 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:52:28.411626 23598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3: Generated new TSK 0
I20260810 00:52:28.412537 23598 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
I20260810 00:52:28.478830 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:33165
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:28.793058 23620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:28.793335 23620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:28.793474 23620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:28.800410 23620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:28.800524 23620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:28.800679 23620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:52:28.809767 23620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:33165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:28.810902 23620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:28.812184 23620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:28.824453 23629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:28.824443 23630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:28.824481 23632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:28.825223 23620 server_base.cc:1034] running on GCE node
I20260810 00:52:28.825757 23620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:28.826571 23620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:28.827764 23620 hybrid_clock.cc:648] HybridClock initialized: now 1786323148827711 us; error 45 us; skew 500 ppm
I20260810 00:52:28.828078 23620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:28 dist-test-slave-jlzn krb5kdc[23465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323148, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:28.831012 23620 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:52:28.832340 23620 webserver.cc:460] Webserver started at http://127.10.97.65:45327/ using document root <none> and password file <none>
I20260810 00:52:28.832944 23620 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:28.833040 23620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:28.833299 23620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:28.836058 23620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "5d0cfb95e95e49f0b2a67e91cf08e602"
format_stamp: "Formatted at 2026-08-10 00:52:28 on dist-test-slave-jlzn"
server_key: "3436ca4b04f5c6c7607bee6d7142ef8c"
server_key_iv: "35477feb54bba188159bed009c86a5a8"
server_key_version: "encryptionkey@0"
I20260810 00:52:28.836787 23620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "5d0cfb95e95e49f0b2a67e91cf08e602"
format_stamp: "Formatted at 2026-08-10 00:52:28 on dist-test-slave-jlzn"
server_key: "3436ca4b04f5c6c7607bee6d7142ef8c"
server_key_iv: "35477feb54bba188159bed009c86a5a8"
server_key_version: "encryptionkey@0"
I20260810 00:52:28.841547 23620 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:52:28.844696 23642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:28.845891 23620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:52:28.846041 23620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "5d0cfb95e95e49f0b2a67e91cf08e602"
format_stamp: "Formatted at 2026-08-10 00:52:28 on dist-test-slave-jlzn"
server_key: "3436ca4b04f5c6c7607bee6d7142ef8c"
server_key_iv: "35477feb54bba188159bed009c86a5a8"
server_key_version: "encryptionkey@0"
I20260810 00:52:28.846216 23620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:28.866437 23620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:28.871042 23620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:28.871301 23620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:28.872313 23620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:28.873580 23620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:28.873656 23620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:28.873775 23620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:28.873826 23620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:28.902622 23620 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:43257
I20260810 00:52:28.902688 23804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:43257 every 8 connection(s)
I20260810 00:52:28.903856 23620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:52:28.909071 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 23620
I20260810 00:52:28.909272 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:52:28.909886 10629 external_mini_cluster.cc:1428] Setting key 1e1ce0612edfeced4a51c4475b68c5a6
Aug 10 00:52:28 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323148, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:52:28.923712 23808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:28.906194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:46395 (local address 127.10.97.126:33165)
0810 00:52:28.906617 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:28.906621 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:28.907548 (+ 927us) server_negotiation.cc:407] Connection header received
0810 00:52:28.909335 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:28.909340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:28.909431 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:28.909524 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:28.911016 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:28.912062 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:28.914170 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:28.914557 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:28.914654 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:28.918379 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:28.918384 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:28.918387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:28.918418 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:28.920774 (+ 2356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:28.921532 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:28.921535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:28.921537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:28.921647 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:28.922044 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:28.922047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:28.922049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:28.922297 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:28.922405 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:28.923155 (+ 750us) server_negotiation.cc:299] Negotiation successful
0810 00:52:28.923368 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":316,"thread_start_us":213,"threads_started":1}
I20260810 00:52:28.924412 23809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:28.906726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.65:46395)
0810 00:52:28.907393 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:28.907437 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:28.909058 (+ 1621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:28.909765 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:28.909773 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:28.910294 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:28.910840 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:28.910865 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:28.912222 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:28.912227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:28.914031 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:28.914046 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:28.914772 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:28.914780 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:28.915034 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:28.915676 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:28.915708 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:28.918172 (+ 2464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:28.920902 (+ 2730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:28.920909 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:28.920923 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:28.921440 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:28.921736 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:28.921740 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:28.921743 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:28.921959 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:28.922406 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:28.922413 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:28.922785 (+ 372us) client_negotiation.cc:769] Sending connection context
0810 00:52:28.923011 (+ 226us) client_negotiation.cc:240] Negotiation successful
0810 00:52:28.923368 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":248,"threads_started":1}
I20260810 00:52:28.925770 23805 heartbeater.cc:347] Connected to a master server at 127.10.97.126:33165
I20260810 00:52:28.926095 23805 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:28.926880 23805 heartbeater.cc:511] Master 127.10.97.126:33165 requested a full tablet report, sending...
I20260810 00:52:28.928978 23533 ts_manager.cc:194] Registered new tserver with Master: 5d0cfb95e95e49f0b2a67e91cf08e602 (127.10.97.65:43257)
I20260810 00:52:28.930321 23533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:46395
WARNING: no policy specified for kudu/127.10.97.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
I20260810 00:52:28.963996 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:33165
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:29.277369 23814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:29.277622 23814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:29.277752 23814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:29.284597 23814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:29.284704 23814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:29.284848 23814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:52:29.294097 23814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:33165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:29.295460 23814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:29.296689 23814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:29.308799 23826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:29.308832 23825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:29.308892 23828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:29.309597 23814 server_base.cc:1034] running on GCE node
I20260810 00:52:29.310146 23814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:29.310984 23814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:29.312181 23814 hybrid_clock.cc:648] HybridClock initialized: now 1786323149312128 us; error 44 us; skew 500 ppm
I20260810 00:52:29.312469 23814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:29 dist-test-slave-jlzn krb5kdc[23465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323149, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:29.315356 23814 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:52:29.316694 23814 webserver.cc:460] Webserver started at http://127.10.97.66:37371/ using document root <none> and password file <none>
I20260810 00:52:29.317317 23814 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:29.317433 23814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:29.317713 23814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:29.320510 23814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "745a85985f5f41f1a2fe7b9756ae7606"
format_stamp: "Formatted at 2026-08-10 00:52:29 on dist-test-slave-jlzn"
server_key: "cf2407db77712fe985b2abd5776ca21c"
server_key_iv: "82a189e849c59802b96f01bda33bb215"
server_key_version: "encryptionkey@0"
I20260810 00:52:29.321260 23814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "745a85985f5f41f1a2fe7b9756ae7606"
format_stamp: "Formatted at 2026-08-10 00:52:29 on dist-test-slave-jlzn"
server_key: "cf2407db77712fe985b2abd5776ca21c"
server_key_iv: "82a189e849c59802b96f01bda33bb215"
server_key_version: "encryptionkey@0"
I20260810 00:52:29.326081 23814 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:52:29.329532 23838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:29.330780 23814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 00:52:29.330931 23814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "745a85985f5f41f1a2fe7b9756ae7606"
format_stamp: "Formatted at 2026-08-10 00:52:29 on dist-test-slave-jlzn"
server_key: "cf2407db77712fe985b2abd5776ca21c"
server_key_iv: "82a189e849c59802b96f01bda33bb215"
server_key_version: "encryptionkey@0"
I20260810 00:52:29.331110 23814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:29.358940 23814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:29.362638 23814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:29.362895 23814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:29.363837 23814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:29.365097 23814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:29.365183 23814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:29.365304 23814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:29.365370 23814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:29.394810 23814 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:41817
I20260810 00:52:29.394892 24000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:41817 every 8 connection(s)
I20260810 00:52:29.396083 23814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:52:29.402834 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 23814
I20260810 00:52:29.403054 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:52:29.403502 10629 external_mini_cluster.cc:1428] Setting key e50e2df15d5b05c3af9881ff5d468836
Aug 10 00:52:29 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323149, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:52:29.415421 23808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:29.398409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:57667 (local address 127.10.97.126:33165)
0810 00:52:29.398585 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:29.398590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:29.399675 (+ 1085us) server_negotiation.cc:407] Connection header received
0810 00:52:29.401449 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:29.401455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:29.401533 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:29.401619 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:29.403276 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:29.404147 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:29.406035 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:29.406442 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:29.406564 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:29.410105 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:29.410109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:29.410113 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:29.410145 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:29.412592 (+ 2447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:29.413407 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:29.413411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:29.413414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:29.413551 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:29.413974 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:29.413977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:29.413979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:29.414212 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:29.414318 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:29.415015 (+ 697us) server_negotiation.cc:299] Negotiation successful
0810 00:52:29.415184 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 00:52:29.416529 24005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:29.398857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.66:57667)
0810 00:52:29.399518 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:29.399555 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:29.401197 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:29.401858 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:29.401866 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:29.402348 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:29.403084 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:29.403111 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:29.404320 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:29.404324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:29.405893 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:29.405906 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:29.406657 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:29.406664 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:29.406889 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:29.407511 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:29.407532 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:29.409885 (+ 2353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:29.412730 (+ 2845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:29.412738 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:29.412751 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:29.413279 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:29.413652 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:29.413657 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:29.413660 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:29.413887 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:29.414319 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:29.414327 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:29.414692 (+ 365us) client_negotiation.cc:769] Sending connection context
0810 00:52:29.414952 (+ 260us) client_negotiation.cc:240] Negotiation successful
0810 00:52:29.415408 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":271,"threads_started":1}
I20260810 00:52:29.417899 24001 heartbeater.cc:347] Connected to a master server at 127.10.97.126:33165
I20260810 00:52:29.418161 24001 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:29.418843 24001 heartbeater.cc:511] Master 127.10.97.126:33165 requested a full tablet report, sending...
I20260810 00:52:29.420261 23533 ts_manager.cc:194] Registered new tserver with Master: 745a85985f5f41f1a2fe7b9756ae7606 (127.10.97.66:41817)
I20260810 00:52:29.421330 23533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:57667
WARNING: no policy specified for kudu/127.10.97.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
I20260810 00:52:29.458158 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:33165
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:29.770762 24013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:29.771034 24013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:29.771137 24013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:29.779335 24013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:29.779459 24013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:29.779623 24013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:52:29.789278 24013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:33165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:29.790572 24013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:29.791895 24013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:29.804139 24025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:29.804103 24030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:29.804179 24027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:29.805145 24013 server_base.cc:1034] running on GCE node
I20260810 00:52:29.805693 24013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:29.806525 24013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:29.807710 24013 hybrid_clock.cc:648] HybridClock initialized: now 1786323149807664 us; error 37 us; skew 500 ppm
I20260810 00:52:29.808017 24013 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:29 dist-test-slave-jlzn krb5kdc[23465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323149, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:29.811304 24013 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:52:29.812615 24013 webserver.cc:460] Webserver started at http://127.10.97.67:36879/ using document root <none> and password file <none>
I20260810 00:52:29.813233 24013 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:29.813345 24013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:29.813620 24013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:29.816404 24013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "3b274e5a02784ab691bc81a74e7aa473"
format_stamp: "Formatted at 2026-08-10 00:52:29 on dist-test-slave-jlzn"
server_key: "c62ff6488fdc2c89c2be34c1185a1fea"
server_key_iv: "300c73f1bb23b7551aea5cfa291c5b21"
server_key_version: "encryptionkey@0"
I20260810 00:52:29.817153 24013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "3b274e5a02784ab691bc81a74e7aa473"
format_stamp: "Formatted at 2026-08-10 00:52:29 on dist-test-slave-jlzn"
server_key: "c62ff6488fdc2c89c2be34c1185a1fea"
server_key_iv: "300c73f1bb23b7551aea5cfa291c5b21"
server_key_version: "encryptionkey@0"
I20260810 00:52:29.822003 24013 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 00:52:29.825191 24037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:29.826406 24013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 00:52:29.826548 24013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "3b274e5a02784ab691bc81a74e7aa473"
format_stamp: "Formatted at 2026-08-10 00:52:29 on dist-test-slave-jlzn"
server_key: "c62ff6488fdc2c89c2be34c1185a1fea"
server_key_iv: "300c73f1bb23b7551aea5cfa291c5b21"
server_key_version: "encryptionkey@0"
I20260810 00:52:29.826721 24013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:29.842739 24013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:29.846642 24013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:29.846889 24013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:29.847769 24013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:29.849043 24013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:29.849128 24013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:29.849257 24013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:29.849308 24013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:29.878599 24013 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:37197
I20260810 00:52:29.878638 24212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:37197 every 8 connection(s)
I20260810 00:52:29.879769 24013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:52:29.887305 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 24013
I20260810 00:52:29.887485 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:52:29.888041 10629 external_mini_cluster.cc:1428] Setting key ec05dc62a5f606a3e8941eeb327035c0
Aug 10 00:52:29 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323149, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:52:29.898455 23808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:29.882131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:55507 (local address 127.10.97.126:33165)
0810 00:52:29.882333 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:29.882339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:29.883505 (+ 1166us) server_negotiation.cc:407] Connection header received
0810 00:52:29.885286 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:29.885291 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:29.885358 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:29.885436 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:29.886783 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:29.887573 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:29.889592 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:29.890089 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:29.890211 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:29.893502 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:29.893506 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:29.893510 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:29.893543 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:29.896128 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:29.896711 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:29.896715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:29.896717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:29.896837 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:29.897165 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:29.897168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:29.897170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:29.897406 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:29.897530 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:29.898055 (+ 525us) server_negotiation.cc:299] Negotiation successful
0810 00:52:29.898210 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 00:52:29.899256 24218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:29.882640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.67:55507)
0810 00:52:29.883339 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:29.883387 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:29.885025 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:29.885618 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:29.885625 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:29.886100 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:29.886655 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:29.886673 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:29.887731 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:29.887737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:29.889462 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:29.889477 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:29.890298 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:29.890303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:29.890492 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:29.891174 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:29.891203 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:29.893331 (+ 2128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:29.896233 (+ 2902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:29.896239 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:29.896249 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:29.896624 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:29.896911 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:29.896914 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:29.896916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:29.897086 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:29.897492 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:29.897497 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:29.897763 (+ 266us) client_negotiation.cc:769] Sending connection context
0810 00:52:29.897962 (+ 199us) client_negotiation.cc:240] Negotiation successful
0810 00:52:29.898302 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":262,"threads_started":1}
I20260810 00:52:29.900404 24214 heartbeater.cc:347] Connected to a master server at 127.10.97.126:33165
I20260810 00:52:29.900664 24214 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:29.901329 24214 heartbeater.cc:511] Master 127.10.97.126:33165 requested a full tablet report, sending...
I20260810 00:52:29.902844 23533 ts_manager.cc:194] Registered new tserver with Master: 3b274e5a02784ab691bc81a74e7aa473 (127.10.97.67:37197)
I20260810 00:52:29.903431 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:52:29.903746 23533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:55507
WARNING: no policy specified for oryx/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.97.65@KRBTEST.COM" created.
I20260810 00:52:29.933511 23805 heartbeater.cc:503] Master 127.10.97.126:33165 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.65.keytab.
Entry for principal oryx/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.65.keytab.
WARNING: no policy specified for oryx/127.10.97.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.97.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.66.keytab.
Entry for principal oryx/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.66.keytab.
WARNING: no policy specified for oryx/127.10.97.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.97.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.67.keytab.
Entry for principal oryx/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.67.keytab.
I20260810 00:52:30.041554 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 23620
Aug 10 00:52:30 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:52:30.056488 24229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:30.042551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45492 (local address 127.10.97.65:43257)
0810 00:52:30.043151 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:30.043158 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:30.043186 (+ 28us) server_negotiation.cc:407] Connection header received
0810 00:52:30.043249 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:30.043256 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:30.043465 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:30.043598 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:30.044892 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:30.045917 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:30.047052 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:30.047601 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:30.047703 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:30.051593 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:30.051604 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:30.051617 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:30.051656 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:30.054147 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:30.054794 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:30.054800 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:30.054802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:30.054927 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:30.055195 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:30.055198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:30.055200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:30.055514 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:30.055742 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:30.055943 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 00:52:30.056139 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":274,"threads_started":1}
I20260810 00:52:30.057669 23799 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:30.137656 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 23620
I20260810 00:52:30.158445 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 23814
Aug 10 00:52:30 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.66@KRBTEST.COM
I20260810 00:52:30.173583 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:30.159388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50504 (local address 127.10.97.66:41817)
0810 00:52:30.159918 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:30.159925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:30.159951 (+ 26us) server_negotiation.cc:407] Connection header received
0810 00:52:30.160012 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:30.160018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:30.160237 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:30.160360 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:30.161498 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:30.162672 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:30.163963 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:30.164495 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:30.164586 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:30.168475 (+ 3889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:30.168487 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:30.168505 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:30.168541 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:30.171085 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:30.171673 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:30.171679 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:30.171682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:30.171797 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:30.172221 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:30.172224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:30.172226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:30.172580 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:30.172814 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:30.173054 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 00:52:30.173259 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":255,"threads_started":1}
I20260810 00:52:30.174823 23995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:30.253980 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 23814
I20260810 00:52:30.272047 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 24013
Aug 10 00:52:30 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.67@KRBTEST.COM
I20260810 00:52:30.287995 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:30.273014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34084 (local address 127.10.97.67:37197)
0810 00:52:30.273631 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:30.273639 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:30.273669 (+ 30us) server_negotiation.cc:407] Connection header received
0810 00:52:30.273732 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:30.273739 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:30.273944 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:30.274090 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:30.275467 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:30.276565 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:30.277828 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:30.278391 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:30.278507 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:30.282728 (+ 4221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:30.282739 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:30.282754 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:30.282787 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:30.285304 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:30.285933 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:30.285940 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:30.285942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:30.286080 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:30.286424 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:30.286428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:30.286430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:30.286780 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:30.287081 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:30.287361 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 00:52:30.287618 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":299,"threads_started":1}
I20260810 00:52:30.289252 24204 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:30.375797 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 24013
I20260810 00:52:30.398046 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 23483
I20260810 00:52:30.399220 23572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:30.499982 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 23483
I20260810 00:52:30.523576 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:33165
--webserver_interface=127.10.97.126
--webserver_port=45083
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:33165
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:30.843540 24242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:30.843861 24242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:30.843981 24242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:30.850878 24242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:52:30.850971 24242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:30.851024 24242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:30.851079 24242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:52:30.851123 24242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:52:30.860491 24242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:33165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:33165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=45083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:30.861637 24242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:30.862851 24242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:30.876386 24248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:30.876420 24249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:30.876441 24251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:30.877270 24242 server_base.cc:1034] running on GCE node
I20260810 00:52:30.877965 24242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:30.879189 24242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:30.880384 24242 hybrid_clock.cc:648] HybridClock initialized: now 1786323150880333 us; error 39 us; skew 500 ppm
I20260810 00:52:30.880681 24242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:30 dist-test-slave-jlzn krb5kdc[23465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323150, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:30.883487 24242 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:52:30.884881 24242 webserver.cc:460] Webserver started at http://127.10.97.126:45083/ using document root <none> and password file <none>
I20260810 00:52:30.885587 24242 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:30.885744 24242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:30.890908 24242 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 00:52:30.893553 24259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:30.894768 24242 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:52:30.894914 24242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "92b3cdd88e3342f9a66bef302a68d7e3"
format_stamp: "Formatted at 2026-08-10 00:52:28 on dist-test-slave-jlzn"
server_key: "750efbed8b8106e79cb65bdbb8b2684a"
server_key_iv: "0e67be58d797322fdc9a063656f70a22"
server_key_version: "encryptionkey@0"
I20260810 00:52:30.895855 24242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:30.913113 24242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:30.916621 24242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:30.916833 24242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:30.933247 24242 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:33165
I20260810 00:52:30.933265 24337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:33165 every 8 connection(s)
I20260810 00:52:30.934623 24242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:52:30.941118 24341 sys_catalog.cc:263] Verifying existing consensus state
I20260810 00:52:30.944167 24341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3: Bootstrap starting.
I20260810 00:52:30.945135 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 24242
I20260810 00:52:30.946125 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:43257
--local_ip_for_outbound_sockets=127.10.97.65
--tserver_master_addrs=127.10.97.126:33165
--webserver_port=45327
--webserver_interface=127.10.97.65
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.97.65
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:52:30.965144 24341 log.cc:824] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:30.973618 24341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 00:52:30.974092 24341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3: Bootstrap complete.
I20260810 00:52:30.982767 24341 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b3cdd88e3342f9a66bef302a68d7e3" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33165 } }
I20260810 00:52:30.983580 24341 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92b3cdd88e3342f9a66bef302a68d7e3, State: Initialized, Role: FOLLOWER
I20260810 00:52:30.984221 24341 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b3cdd88e3342f9a66bef302a68d7e3" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33165 } }
I20260810 00:52:30.984443 24341 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:52:30.984558 24341 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:52:30.984722 24341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:52:30.988389 24341 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b3cdd88e3342f9a66bef302a68d7e3" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33165 } }
I20260810 00:52:30.988852 24341 leader_election.cc:302] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92b3cdd88e3342f9a66bef302a68d7e3; no voters:
I20260810 00:52:30.989428 24341 leader_election.cc:288] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 00:52:30.989585 24347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 00:52:30.990265 24347 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [term 2 LEADER]: Becoming Leader. State: Replica: 92b3cdd88e3342f9a66bef302a68d7e3, State: Running, Role: LEADER
I20260810 00:52:30.990834 24347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b3cdd88e3342f9a66bef302a68d7e3" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33165 } }
I20260810 00:52:30.991430 24341 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:52:30.993415 24348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92b3cdd88e3342f9a66bef302a68d7e3. Latest consensus state: current_term: 2 leader_uuid: "92b3cdd88e3342f9a66bef302a68d7e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b3cdd88e3342f9a66bef302a68d7e3" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33165 } } }
I20260810 00:52:30.993487 24349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "92b3cdd88e3342f9a66bef302a68d7e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b3cdd88e3342f9a66bef302a68d7e3" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33165 } } }
I20260810 00:52:30.993769 24348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:30.993767 24349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:30.994659 24356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:52:31.011548 24356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:52:31.013070 24356 catalog_manager.cc:1259] Loaded cluster ID: bc8587967ecb45a882b68a080563b1da
I20260810 00:52:31.013222 24356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:52:31.017166 24356 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:52:31.018620 24356 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 92b3cdd88e3342f9a66bef302a68d7e3: Loaded TSK: 0
I20260810 00:52:31.020219 24356 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 00:52:31.281013 24343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:31.281294 24343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:31.281430 24343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:31.288385 24343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:31.288538 24343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:31.288736 24343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:52:31.298465 24343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.65.keytab
--principal=oryx/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:43257
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=45327
--tserver_master_addrs=127.10.97.126:33165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:31.299695 24343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:31.300966 24343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:31.314693 24385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:31.314746 24381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:31.314890 24379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:31.315796 24343 server_base.cc:1034] running on GCE node
I20260810 00:52:31.316393 24343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:31.317251 24343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:31.318444 24343 hybrid_clock.cc:648] HybridClock initialized: now 1786323151318394 us; error 44 us; skew 500 ppm
I20260810 00:52:31.318735 24343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:31 dist-test-slave-jlzn krb5kdc[23465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323151, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:31.321869 24343 init.cc:377] Logged in from keytab as oryx/127.10.97.65@KRBTEST.COM (short username oryx)
I20260810 00:52:31.323267 24343 webserver.cc:460] Webserver started at http://127.10.97.65:45327/ using document root <none> and password file <none>
I20260810 00:52:31.323939 24343 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:31.324079 24343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:31.329281 24343 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 00:52:31.331795 24393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:31.333056 24343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:52:31.333210 24343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "5d0cfb95e95e49f0b2a67e91cf08e602"
format_stamp: "Formatted at 2026-08-10 00:52:28 on dist-test-slave-jlzn"
server_key: "3436ca4b04f5c6c7607bee6d7142ef8c"
server_key_iv: "35477feb54bba188159bed009c86a5a8"
server_key_version: "encryptionkey@0"
I20260810 00:52:31.334079 24343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:31.356719 24343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:31.360183 24343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:31.360414 24343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:31.361271 24343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:31.362489 24343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:31.362578 24343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:31.362679 24343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:31.362751 24343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:31.392158 24343 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:43257
I20260810 00:52:31.392241 24554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:43257 every 8 connection(s)
I20260810 00:52:31.393535 24343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:52:31.402895 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 24343
I20260810 00:52:31.403502 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:41817
--local_ip_for_outbound_sockets=127.10.97.66
--tserver_master_addrs=127.10.97.126:33165
--webserver_port=37371
--webserver_interface=127.10.97.66
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.97.66
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 00:52:31 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:31 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 00:52:31.723210 24563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:31.723460 24563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:31.723575 24563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:31.730500 24563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:31.730600 24563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:31.730753 24563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:52:31.740061 24563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.66.keytab
--principal=oryx/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:41817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=37371
--tserver_master_addrs=127.10.97.126:33165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:31.741214 24563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:31.742425 24563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:31.757709 24572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:31.757907 24573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:31.758663 24576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:31.758810 24563 server_base.cc:1034] running on GCE node
I20260810 00:52:31.759423 24563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:31.760372 24563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:31.761576 24563 hybrid_clock.cc:648] HybridClock initialized: now 1786323151761516 us; error 57 us; skew 500 ppm
I20260810 00:52:31.761880 24563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:31 dist-test-slave-jlzn krb5kdc[23465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323151, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:31.769426 24563 init.cc:377] Logged in from keytab as oryx/127.10.97.66@KRBTEST.COM (short username oryx)
I20260810 00:52:31.770952 24563 webserver.cc:460] Webserver started at http://127.10.97.66:37371/ using document root <none> and password file <none>
I20260810 00:52:31.771613 24563 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:31.771747 24563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:31.776980 24563 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 00:52:31.779510 24585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:31.780795 24563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 00:52:31.780936 24563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "745a85985f5f41f1a2fe7b9756ae7606"
format_stamp: "Formatted at 2026-08-10 00:52:29 on dist-test-slave-jlzn"
server_key: "cf2407db77712fe985b2abd5776ca21c"
server_key_iv: "82a189e849c59802b96f01bda33bb215"
server_key_version: "encryptionkey@0"
I20260810 00:52:31.781826 24563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:31.796557 24563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:31.800366 24563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:31.800594 24563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:31.801535 24563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:31.802860 24563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:31.802953 24563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:31.803049 24563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:31.803117 24563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:31.841636 24563 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:41817
I20260810 00:52:31.841684 24762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:41817 every 8 connection(s)
I20260810 00:52:31.843014 24563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:52:31.848407 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 24563
I20260810 00:52:31.849215 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:37197
--local_ip_for_outbound_sockets=127.10.97.67
--tserver_master_addrs=127.10.97.126:33165
--webserver_port=36879
--webserver_interface=127.10.97.67
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.97.67
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 00:52:31 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:31 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 00:52:32.263175 24767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:32.263604 24767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:32.263773 24767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:32.276351 24767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:32.276508 24767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:32.276783 24767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:52:32.293490 24767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/krb5kdc/oryx_127.10.97.67.keytab
--principal=oryx/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:37197
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=36879
--tserver_master_addrs=127.10.97.126:33165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:32.295393 24767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:32.297310 24767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:32.316617 24773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:32.320151 24778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:32.325585 24780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:32.325991 24767 server_base.cc:1034] running on GCE node
I20260810 00:52:32.326764 24767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:32.328050 24767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:32.329303 24767 hybrid_clock.cc:648] HybridClock initialized: now 1786323152329239 us; error 52 us; skew 500 ppm
I20260810 00:52:32.329700 24767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:32 dist-test-slave-jlzn krb5kdc[23465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323152, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:32.334455 24767 init.cc:377] Logged in from keytab as oryx/127.10.97.67@KRBTEST.COM (short username oryx)
I20260810 00:52:32.336637 24767 webserver.cc:460] Webserver started at http://127.10.97.67:36879/ using document root <none> and password file <none>
I20260810 00:52:32.337817 24767 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:32.337987 24767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:32.346318 24767 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 00:52:32.350198 24787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:32.352382 24767 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 00:52:32.352591 24767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "3b274e5a02784ab691bc81a74e7aa473"
format_stamp: "Formatted at 2026-08-10 00:52:29 on dist-test-slave-jlzn"
server_key: "c62ff6488fdc2c89c2be34c1185a1fea"
server_key_iv: "300c73f1bb23b7551aea5cfa291c5b21"
server_key_version: "encryptionkey@0"
I20260810 00:52:32.354104 24767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:32.372114 24767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:32.377453 24767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:32.377712 24767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:32.378789 24767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:32.380501 24767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:32.380653 24767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:32.380769 24767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:32.380837 24767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:32.456161 24767 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:37197
I20260810 00:52:32.456339 24959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:37197 every 8 connection(s)
I20260810 00:52:32.458113 24767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:52:32.464998 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 24767
Aug 10 00:52:32 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:32 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:32.500308 24966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:32.466654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37948 (local address 127.10.97.126:33165)
0810 00:52:32.478178 (+ 11524us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:32.478182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:32.478196 (+ 14us) server_negotiation.cc:407] Connection header received
0810 00:52:32.478240 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:32.478243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:32.478297 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:32.478387 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:32.480553 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:32.482055 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:32.484192 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:32.484881 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:32.485028 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:32.489434 (+ 4406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:32.489447 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:32.489471 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:32.489519 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:32.493956 (+ 4437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:32.494917 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:32.494928 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:32.494939 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:32.495209 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:32.495710 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:32.495715 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:32.495719 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:32.496444 (+ 725us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:32.496792 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:32.498292 (+ 1500us) server_negotiation.cc:299] Negotiation successful
0810 00:52:32.498806 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11419,"thread_start_us":11297,"threads_started":1}
W20260810 00:52:32.522956 24562 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.117s user 0.005s sys 0.009s
W20260810 00:52:32.524312 24561 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:31.396220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:37203 (local address 127.10.97.126:33165)
0810 00:52:31.396860 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:31.396871 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:31.397236 (+ 365us) server_negotiation.cc:407] Connection header received
0810 00:52:31.399261 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:31.399274 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:31.399565 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:31.399872 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:31.401252 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:31.402567 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:31.404259 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:31.405214 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:31.405377 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:32.524012 (+1118635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.65:37203: BlockingRecv error: recv got EOF from 127.10.97.65:37203 (error 108)
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":236,"threads_started":1}
I20260810 00:52:32.524876 24562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:31.396316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.65:37203)
0810 00:52:31.397085 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:31.397131 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:31.398783 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:31.400003 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:31.400010 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:31.400562 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:31.401077 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:31.401094 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:31.402696 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:31.402699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:31.404084 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:31.404097 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:31.405394 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:31.405399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:31.405585 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:31.406174 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:31.406196 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:32.523714 (+1117518us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":307,"threads_started":1}
W20260810 00:52:32.525765 24557 heartbeater.cc:644] Failed to heartbeat to 127.10.97.126:33165 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.97.126:33165: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Aug 10 00:52:32 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:32 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:32.535180 24562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:32.526407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.65:37427)
0810 00:52:32.526668 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:32.526687 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:32.526796 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:32.527087 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:32.527092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:32.527454 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:32.527612 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:32.527621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:32.529014 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:32.529020 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:32.530119 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:32.530135 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:32.530821 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:32.530829 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:32.530983 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:32.531547 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:32.531565 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:32.534924 (+ 3359us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145}
W20260810 00:52:32.535337 24561 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:32.526550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:37427 (local address 127.10.97.126:33165)
0810 00:52:32.526708 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:32.526713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:32.526733 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:52:32.526908 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:32.526912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:32.526968 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:32.527043 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:32.527755 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:32.528845 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:32.530260 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:32.530657 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:32.530769 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:32.535117 (+ 4348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.65:37427: BlockingRecv error: recv got EOF from 127.10.97.65:37427 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 10 00:52:32 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:32 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:32.544508 24562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:32.536061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.65:34095)
0810 00:52:32.536366 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:32.536380 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:32.536478 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:32.536770 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:32.536775 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:32.537077 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:32.537222 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:32.537232 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:32.538234 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:32.538239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:32.539191 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:32.539204 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:32.540453 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:32.540459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:32.540639 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:32.541266 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:32.541283 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:32.544266 (+ 2983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229}
W20260810 00:52:32.544692 24561 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:32.536149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:34095 (local address 127.10.97.126:33165)
0810 00:52:32.536275 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:32.536279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:32.536448 (+ 169us) server_negotiation.cc:407] Connection header received
0810 00:52:32.536574 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:32.536580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:32.536642 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:32.536764 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:32.537346 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:32.538104 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:32.539325 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:32.540026 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:32.540137 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:32.544468 (+ 4331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.65:34095: BlockingRecv error: recv got EOF from 127.10.97.65:34095 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260810 00:52:32.545161 24557 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 00:52:32.987444 24766 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.133s user 0.002s sys 0.012s
W20260810 00:52:32.989017 24765 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:31.845666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:54611 (local address 127.10.97.126:33165)
0810 00:52:31.846075 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:31.846080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:31.847389 (+ 1309us) server_negotiation.cc:407] Connection header received
0810 00:52:31.849273 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:31.849279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:31.849359 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:31.849455 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:31.850893 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:31.851694 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:31.853226 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:31.853646 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:31.853756 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:32.988769 (+1135013us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.66:54611: BlockingRecv error: recv got EOF from 127.10.97.66:54611 (error 108)
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":204,"threads_started":1}
I20260810 00:52:32.990095 24766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:31.846434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.66:54611)
0810 00:52:31.847242 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:31.847279 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:31.849007 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:31.849696 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:31.849703 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:31.850190 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:31.850727 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:31.850744 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:31.851863 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:31.851867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:31.853079 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:31.853092 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:31.853859 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:31.853867 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:31.854082 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:31.854697 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:31.854718 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:32.988570 (+1133852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":314,"threads_started":1}
W20260810 00:52:32.991114 24763 heartbeater.cc:644] Failed to heartbeat to 127.10.97.126:33165 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.97.126:33165: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Aug 10 00:52:32 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:32 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:33.001194 24766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:32.991757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.66:54897)
0810 00:52:32.992030 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:32.992054 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:32.992215 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:32.992511 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:32.992516 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:32.993126 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:32.993381 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:32.993393 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:32.994746 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:32.994751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:32.995764 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:32.995778 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:32.996504 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:32.996520 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:32.996669 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:32.997315 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:32.997331 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:33.000930 (+ 3599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":79}
W20260810 00:52:33.001354 24765 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:32.991857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:54897 (local address 127.10.97.126:33165)
0810 00:52:32.992037 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:32.992042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:32.992125 (+ 83us) server_negotiation.cc:407] Connection header received
0810 00:52:32.992323 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:32.992326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:32.992387 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:32.992463 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:32.993534 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:32.994619 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:32.995916 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:32.996341 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:32.996447 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:33.001137 (+ 4690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.66:54897: BlockingRecv error: recv got EOF from 127.10.97.66:54897 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 00:52:33 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:33 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:33.010257 24766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:33.001966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.66:45081)
0810 00:52:33.002103 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:33.002127 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:33.002205 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:33.002635 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:33.002640 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:33.002928 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:33.003072 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:33.003080 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:33.004072 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:33.004078 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:33.004996 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:33.005008 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:33.005881 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:33.005886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:33.006070 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:33.006653 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:33.006670 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:33.010016 (+ 3346us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 00:52:33.010440 24765 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:33.002227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:45081 (local address 127.10.97.126:33165)
0810 00:52:33.002385 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:33.002389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:33.002402 (+ 13us) server_negotiation.cc:407] Connection header received
0810 00:52:33.002437 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:33.002440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:33.002502 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:33.002582 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:33.003213 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:33.003942 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:33.005146 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:33.005743 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:33.005845 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:33.010222 (+ 4377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.66:45081: BlockingRecv error: recv got EOF from 127.10.97.66:45081 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 00:52:33.010679 24763 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 00:52:33.515842 24963 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.029s user 0.002s sys 0.015s
W20260810 00:52:33.516927 24964 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:32.463570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:39387 (local address 127.10.97.126:33165)
0810 00:52:32.476225 (+ 12655us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:32.476232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:32.476255 (+ 23us) server_negotiation.cc:407] Connection header received
0810 00:52:32.476328 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:32.476334 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:32.476418 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:32.476534 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:32.480500 (+ 3966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:32.481873 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:32.484649 (+ 2776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:32.485492 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:32.485593 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:33.516737 (+1031144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.67:39387: BlockingRecv error: recv got EOF from 127.10.97.67:39387 (error 108)
Metrics: {"server-negotiator.queue_time_us":12513,"thread_start_us":12375,"threads_started":1}
I20260810 00:52:33.517462 24963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:32.462841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.67:39387)
0810 00:52:32.468450 (+ 5609us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:32.468502 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:32.471176 (+ 2674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:32.478466 (+ 7290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:32.478480 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:32.479319 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:32.480270 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:32.480297 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:32.482090 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:32.482097 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:32.484450 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:32.484472 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:32.485945 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:32.485953 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:32.486255 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:32.486936 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:32.486965 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:33.516556 (+1029591us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":5245,"thread_start_us":9147,"threads_started":1}
W20260810 00:52:33.518378 24960 heartbeater.cc:644] Failed to heartbeat to 127.10.97.126:33165 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.97.126:33165: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Aug 10 00:52:33 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:33 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:33.526938 24963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:33.519014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.67:52935)
0810 00:52:33.519283 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:33.519302 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:33.519422 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:33.519735 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:33.519740 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:33.520123 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:33.520289 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:33.520298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:33.521269 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:33.521275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:33.522241 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:33.522253 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:33.522962 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:33.522969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:33.523139 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:33.523642 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:33.523658 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:33.526682 (+ 3024us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154}
W20260810 00:52:33.527101 24964 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:33.519087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:52935 (local address 127.10.97.126:33165)
0810 00:52:33.519256 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:33.519263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:33.519368 (+ 105us) server_negotiation.cc:407] Connection header received
0810 00:52:33.519532 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:33.519538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:33.519603 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:33.519688 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:33.520422 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:33.521137 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:33.522373 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:33.522800 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:33.522905 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:33.526888 (+ 3983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.67:52935: BlockingRecv error: recv got EOF from 127.10.97.67:52935 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 10 00:52:33 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:33 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:33.534641 24963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:33.527573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.67:41669)
0810 00:52:33.527707 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:33.527721 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:33.527837 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:33.528154 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:33.528159 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:33.528430 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:33.528563 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:33.528571 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:33.529482 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:33.529485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:33.530404 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:33.530415 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:33.531117 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:33.531122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:33.531285 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:33.531725 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:33.531740 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:33.534458 (+ 2718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 00:52:33.534853 24964 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:33.527662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:41669 (local address 127.10.97.126:33165)
0810 00:52:33.527907 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:33.527912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:33.527927 (+ 15us) server_negotiation.cc:407] Connection header received
0810 00:52:33.527968 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:33.527973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:33.528022 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:33.528095 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:33.528694 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:33.529347 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:33.530533 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:33.530978 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:33.531081 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:33.534619 (+ 3538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.67:41669: BlockingRecv error: recv got EOF from 127.10.97.67:41669 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
W20260810 00:52:33.534894 24960 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 00:52:33 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:33 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:33.554272 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:33.545627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.65:58105)
0810 00:52:33.546133 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:33.546149 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:33.546259 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:33.546503 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:33.546506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:33.546913 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:33.547061 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:33.547069 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:33.548062 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:33.548067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:33.549015 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:33.549028 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:33.549749 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:33.549755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:33.549894 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:33.550433 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:33.550451 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:33.553955 (+ 3504us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":290,"threads_started":1}
W20260810 00:52:33.554405 24964 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:33.545691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:58105 (local address 127.10.97.126:33165)
0810 00:52:33.545891 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:33.545898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:33.546216 (+ 318us) server_negotiation.cc:407] Connection header received
0810 00:52:33.546341 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:33.546344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:33.546403 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:33.546478 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:33.547185 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:33.547907 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:33.549149 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:33.549565 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:33.549669 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:33.554182 (+ 4513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.65:58105: BlockingRecv error: recv got EOF from 127.10.97.65:58105 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 00:52:34 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:34 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:34.020056 24969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:34.011231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.66:34995)
0810 00:52:34.011774 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:34.011792 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:34.011929 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:34.012197 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:34.012200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:34.012621 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:34.012765 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:34.012773 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:34.013779 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:34.013783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:34.014736 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:34.014751 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:34.015412 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:34.015416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:34.015530 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:34.016134 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:34.016153 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:34.019689 (+ 3536us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":307,"threads_started":1}
W20260810 00:52:34.020151 24964 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:34.011285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:34995 (local address 127.10.97.126:33165)
0810 00:52:34.011468 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:34.011472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:34.011886 (+ 414us) server_negotiation.cc:407] Connection header received
0810 00:52:34.012021 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:34.012025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:34.012088 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:34.012167 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:34.012894 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:34.013664 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:34.014867 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:34.015270 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:34.015369 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:34.019938 (+ 4569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.66:34995: BlockingRecv error: recv got EOF from 127.10.97.66:34995 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 00:52:34 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:34 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:34.543947 24970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:34.535396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.67:51435)
0810 00:52:34.535928 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:34.535944 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:34.536054 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:34.536342 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:34.536346 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:34.536794 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:34.536950 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:34.536959 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:34.537979 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:34.537983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:34.538918 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:34.538931 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:34.539589 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:34.539593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:34.539709 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:34.540258 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:34.540275 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:34.543666 (+ 3391us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":286,"threads_started":1}
W20260810 00:52:34.544090 24972 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:34.535472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:51435 (local address 127.10.97.126:33165)
0810 00:52:34.535900 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:34.535904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:34.536003 (+ 99us) server_negotiation.cc:407] Connection header received
0810 00:52:34.536150 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:34.536154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:34.536225 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:34.536312 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:34.537075 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:34.537871 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:34.539022 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:34.539459 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:34.539544 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:34.543885 (+ 4341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.67:51435: BlockingRecv error: recv got EOF from 127.10.97.67:51435 (error 108)
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":214,"threads_started":1}
Aug 10 00:52:34 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:34 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:34.562733 24973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:34.555035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.65:46365)
0810 00:52:34.555523 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:34.555538 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:34.555651 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:34.555924 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:34.555927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:34.556321 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:34.556492 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:34.556501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:34.557391 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:34.557394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:34.558291 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:34.558302 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:34.558870 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:34.558874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:34.559010 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:34.559471 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:34.559487 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:34.562492 (+ 3005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":256,"threads_started":1}
W20260810 00:52:34.562830 24972 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:34.555091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:46365 (local address 127.10.97.126:33165)
0810 00:52:34.555270 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:34.555275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:34.555602 (+ 327us) server_negotiation.cc:407] Connection header received
0810 00:52:34.555734 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:34.555737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:34.555789 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:34.555894 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:34.556612 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:34.557295 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:34.558387 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:34.558766 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:34.558845 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:34.562647 (+ 3802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.65:46365: BlockingRecv error: recv got EOF from 127.10.97.65:46365 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 00:52:35 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:35 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:35.029920 24974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:35.020978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.66:35567)
0810 00:52:35.021476 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:35.021492 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:35.021621 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:35.021932 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:35.021937 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:35.022372 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:35.022560 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:35.022569 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:35.023545 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:35.023550 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:35.024553 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:35.024569 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:35.025426 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:35.025430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:35.025600 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:35.026223 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:35.026241 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:35.029610 (+ 3369us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":288,"threads_started":1}
W20260810 00:52:35.030100 24972 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:35.021064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:35567 (local address 127.10.97.126:33165)
0810 00:52:35.021247 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:35.021253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:35.021576 (+ 323us) server_negotiation.cc:407] Connection header received
0810 00:52:35.021731 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:35.021736 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:35.021804 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:35.021883 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:35.022712 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:35.023418 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:35.024698 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:35.025264 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:35.025367 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:35.029845 (+ 4478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.66:35567: BlockingRecv error: recv got EOF from 127.10.97.66:35567 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 10 00:52:35 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:35 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:35.553205 24976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:35.544784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.67:49945)
0810 00:52:35.545282 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:35.545298 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:35.545401 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:35.545687 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:35.545690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:35.546065 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:35.546261 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:35.546270 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:35.547183 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:35.547187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:35.548177 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:35.548190 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:35.548802 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:35.548806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:35.548919 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:35.549457 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:35.549475 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:35.552958 (+ 3483us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":244,"threads_started":1}
W20260810 00:52:35.553361 24977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:35.544928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:49945 (local address 127.10.97.126:33165)
0810 00:52:35.545362 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:35.545366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:35.545381 (+ 15us) server_negotiation.cc:407] Connection header received
0810 00:52:35.545503 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:35.545506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:35.545573 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:35.545667 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:35.546363 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:35.547079 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:35.548283 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:35.548692 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:35.548780 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:35.553140 (+ 4360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.67:49945: BlockingRecv error: recv got EOF from 127.10.97.67:49945 (error 108)
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":222,"threads_started":1}
Aug 10 00:52:35 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:35 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:35.570873 24978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:35.563505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.65:46947)
0810 00:52:35.564022 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:35.564037 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:35.564142 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:35.564367 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:35.564370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:35.564717 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:35.564872 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:35.564880 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:35.565750 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:35.565753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:35.566646 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:35.566657 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:35.567230 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:35.567234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:35.567352 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:35.567874 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:35.567891 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:35.570664 (+ 2773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":281,"threads_started":1}
W20260810 00:52:35.571008 24977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:35.563565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:46947 (local address 127.10.97.126:33165)
0810 00:52:35.563701 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:35.563705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:35.564095 (+ 390us) server_negotiation.cc:407] Connection header received
0810 00:52:35.564215 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:35.564218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:35.564270 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:35.564339 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:35.564980 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:35.565645 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:35.566743 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:35.567133 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:35.567213 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:35.570830 (+ 3617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.65:46947: BlockingRecv error: recv got EOF from 127.10.97.65:46947 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 10 00:52:36 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:36 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:36.040313 24979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:36.031132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.66:33145)
0810 00:52:36.031657 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:36.031675 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:36.031786 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:36.032172 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:36.032178 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:36.032657 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:36.032887 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:36.032900 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:36.034102 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:36.034108 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:36.035331 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:36.035347 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:36.036211 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:36.036215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:36.036341 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:36.036898 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:36.036916 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:36.040081 (+ 3165us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":214,"threads_started":1}
W20260810 00:52:36.040482 24977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:36.031225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:33145 (local address 127.10.97.126:33165)
0810 00:52:36.031451 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:36.031457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:36.031758 (+ 301us) server_negotiation.cc:407] Connection header received
0810 00:52:36.031919 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:36.031925 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:36.032021 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:36.032107 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:36.033048 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:36.033955 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:36.035479 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:36.036082 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:36.036184 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:36.040265 (+ 4081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.66:33145: BlockingRecv error: recv got EOF from 127.10.97.66:33145 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 10 00:52:36 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:36 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:36.562353 24980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:36.553996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.67:45643)
0810 00:52:36.554499 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:36.554516 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:36.554612 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:36.554890 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:36.554893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:36.555282 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:36.555456 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:36.555464 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:36.556397 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:36.556400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:36.557312 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:36.557322 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:36.557920 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:36.557924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:36.558035 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:36.558556 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:36.558573 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:36.562108 (+ 3535us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":258,"threads_started":1}
W20260810 00:52:36.562516 24982 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:36.554060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:45643 (local address 127.10.97.126:33165)
0810 00:52:36.554532 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:36.554536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:36.554569 (+ 33us) server_negotiation.cc:407] Connection header received
0810 00:52:36.554713 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:36.554716 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:36.554784 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:36.554862 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:36.555564 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:36.556289 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:36.557415 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:36.557818 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:36.557906 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:36.562289 (+ 4383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.67:45643: BlockingRecv error: recv got EOF from 127.10.97.67:45643 (error 108)
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":247,"threads_started":1}
Aug 10 00:52:36 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:36 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:36.579384 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:36.571701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.65:43053)
0810 00:52:36.572260 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:36.572275 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:36.572377 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:36.572623 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:36.572634 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:36.572967 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:36.573125 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:36.573136 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:36.573967 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:36.573970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:36.574968 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:36.574979 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:36.575579 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:36.575583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:36.575707 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:36.576246 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:36.576263 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:36.579144 (+ 2881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":325,"threads_started":1}
W20260810 00:52:36.579478 24982 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:36.571771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:43053 (local address 127.10.97.126:33165)
0810 00:52:36.571940 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:36.571945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:36.572340 (+ 395us) server_negotiation.cc:407] Connection header received
0810 00:52:36.572456 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:36.572459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:36.572514 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:36.572584 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:36.573228 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:36.573872 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:36.575074 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:36.575477 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:36.575560 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:36.579311 (+ 3751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.65:43053: BlockingRecv error: recv got EOF from 127.10.97.65:43053 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 10 00:52:37 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:37 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:37.055356 24984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:37.041203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.66:58565)
0810 00:52:37.041785 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:37.041805 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:37.041919 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:37.042196 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:37.042199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:37.042594 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:37.042767 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:37.042776 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:37.043697 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:37.043701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:37.044734 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:37.044746 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:37.045337 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:37.045341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:37.045460 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:37.052074 (+ 6614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:37.052092 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:37.055099 (+ 3007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":371,"threads_started":1}
W20260810 00:52:37.055513 24982 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:37.041218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:58565 (local address 127.10.97.126:33165)
0810 00:52:37.041410 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:37.041417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:37.041879 (+ 462us) server_negotiation.cc:407] Connection header received
0810 00:52:37.042017 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:37.042021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:37.042092 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:37.042166 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:37.042871 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:37.043588 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:37.044837 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:37.045233 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:37.045321 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:37.055293 (+ 9972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.66:58565: BlockingRecv error: recv got EOF from 127.10.97.66:58565 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 00:52:37 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:37 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:37.571431 24985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:37.563141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.67:49979)
0810 00:52:37.563609 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:37.563626 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:37.563737 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:37.564060 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:37.564064 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:37.564461 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:37.564623 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:37.564632 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:37.565556 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:37.565559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:37.566575 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:37.566588 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:37.567208 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:37.567214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:37.567364 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:37.567989 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:37.568007 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:37.571187 (+ 3180us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":255,"threads_started":1}
W20260810 00:52:37.571614 24986 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:37.563209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:49979 (local address 127.10.97.126:33165)
0810 00:52:37.563595 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:37.563599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:37.563688 (+ 89us) server_negotiation.cc:407] Connection header received
0810 00:52:37.563851 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:37.563855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:37.563927 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:37.564014 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:37.564731 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:37.565453 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:37.566679 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:37.567065 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:37.567158 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:37.571365 (+ 4207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.67:49979: BlockingRecv error: recv got EOF from 127.10.97.67:49979 (error 108)
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":203,"threads_started":1}
Aug 10 00:52:37 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:37 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:37.588481 24987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:37.580198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.65:47715)
0810 00:52:37.580630 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:37.580646 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:37.580764 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:37.581059 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:37.581063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:37.581473 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:37.581619 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:37.581627 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:37.582557 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:37.582563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:37.583476 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:37.583487 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:37.584195 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:37.584201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:37.584365 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:37.584851 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:37.584868 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:37.588179 (+ 3311us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":217,"threads_started":1}
W20260810 00:52:37.588575 24986 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:37.580195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:47715 (local address 127.10.97.126:33165)
0810 00:52:37.580338 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:37.580343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:37.580725 (+ 382us) server_negotiation.cc:407] Connection header received
0810 00:52:37.580872 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:37.580877 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:37.580937 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:37.581011 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:37.581757 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:37.582433 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:37.583624 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:37.584053 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:37.584157 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:37.588357 (+ 4200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.65:47715: BlockingRecv error: recv got EOF from 127.10.97.65:47715 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 10 00:52:38 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:38 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:38.064497 24988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:38.056148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.66:40935)
0810 00:52:38.056643 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:38.056659 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:38.056768 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:38.057033 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:38.057036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:38.057462 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:38.057620 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:38.057630 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:38.058524 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:38.058528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:38.059522 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:38.059536 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:38.060290 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:38.060296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:38.060475 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:38.061095 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:38.061113 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:38.064280 (+ 3167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":256,"threads_started":1}
W20260810 00:52:38.064673 24986 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:38.056306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:40935 (local address 127.10.97.126:33165)
0810 00:52:38.056452 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:38.056456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:38.056718 (+ 262us) server_negotiation.cc:407] Connection header received
0810 00:52:38.056854 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:38.056857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:38.056926 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:38.057003 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:38.057728 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:38.058422 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:38.059671 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:38.060160 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:38.060281 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:38.064458 (+ 4177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.66:40935: BlockingRecv error: recv got EOF from 127.10.97.66:40935 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 10 00:52:38 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:38 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:38.580650 24990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:38.572301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.67:53423)
0810 00:52:38.572781 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:38.572797 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:38.572910 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:38.573191 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:38.573194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:38.573603 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:38.573760 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:38.573768 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:38.574689 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:38.574694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:38.576080 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:38.576096 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:38.576773 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:38.576778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:38.576899 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:38.577404 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:38.577420 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:38.580421 (+ 3001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":232,"threads_started":1}
W20260810 00:52:38.580871 24992 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:38.572366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:53423 (local address 127.10.97.126:33165)
0810 00:52:38.572784 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:38.572788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:38.572860 (+ 72us) server_negotiation.cc:407] Connection header received
0810 00:52:38.573015 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:38.573019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:38.573083 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:38.573162 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:38.573864 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:38.574561 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:38.576221 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:38.576656 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:38.576747 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:38.580654 (+ 3907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.67:53423: BlockingRecv error: recv got EOF from 127.10.97.67:53423 (error 108)
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":224,"threads_started":1}
Aug 10 00:52:38 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:38 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:38.597012 24993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:38.589359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.65:48195)
0810 00:52:38.589778 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:38.589793 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:38.589896 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:38.590107 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:38.590110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:38.590482 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:38.590646 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:38.590654 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:38.591661 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:38.591664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:38.592675 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:38.592688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:38.593292 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:38.593296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:38.593414 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:38.593876 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:38.593892 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:38.596790 (+ 2898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":210,"threads_started":1}
W20260810 00:52:38.597151 24992 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:38.589403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:48195 (local address 127.10.97.126:33165)
0810 00:52:38.589530 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:38.589534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:38.589852 (+ 318us) server_negotiation.cc:407] Connection header received
0810 00:52:38.589968 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:38.589971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:38.590023 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:38.590091 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:38.590746 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:38.591533 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:38.592781 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:38.593179 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:38.593267 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:38.596960 (+ 3693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.65:48195: BlockingRecv error: recv got EOF from 127.10.97.65:48195 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 10 00:52:39 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:39 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:39.073887 24994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:39.065323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.66:53313)
0810 00:52:39.065810 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:39.065825 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:39.065954 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:39.066282 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:39.066287 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:39.066708 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:39.066879 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:39.066889 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:39.067982 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:39.067986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:39.068973 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:39.068987 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:39.069681 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:39.069686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:39.069843 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:39.070401 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:39.070420 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:39.073566 (+ 3146us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":231,"threads_started":1}
W20260810 00:52:39.074016 24992 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:39.065415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:53313 (local address 127.10.97.126:33165)
0810 00:52:39.065600 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:39.065606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:39.065907 (+ 301us) server_negotiation.cc:407] Connection header received
0810 00:52:39.066063 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:39.066067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:39.066151 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:39.066231 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:39.067024 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:39.067844 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:39.069118 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:39.069524 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:39.069629 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:39.073775 (+ 4146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.66:53313: BlockingRecv error: recv got EOF from 127.10.97.66:53313 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 00:52:39 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:39 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:39.590055 24995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:39.581421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.67:34327)
0810 00:52:39.582035 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:39.582057 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:39.582187 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:39.582528 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:39.582532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:39.583006 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:39.583197 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:39.583212 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:39.584240 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:39.584244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:39.585160 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:39.585171 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:39.585866 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:39.585870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:39.585995 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:39.586541 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:39.586559 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:39.589731 (+ 3172us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":238,"threads_started":1}
W20260810 00:52:39.590202 24996 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:39.581502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:34327 (local address 127.10.97.126:33165)
0810 00:52:39.582171 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:39.582177 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:39.582197 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:52:39.582314 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:39.582319 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:39.582389 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:39.582476 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:39.583340 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:39.584136 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:39.585298 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:39.585739 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:39.585829 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:39.589938 (+ 4109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.67:34327: BlockingRecv error: recv got EOF from 127.10.97.67:34327 (error 108)
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":474,"threads_started":1}
Aug 10 00:52:39 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:39 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:39.606005 24998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:39.597754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.65:53859)
0810 00:52:39.598275 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:39.598290 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:39.598404 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:39.598642 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:39.598645 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:39.598998 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:39.599186 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:39.599199 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:39.600146 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:39.600150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:39.601168 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:39.601182 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:39.601781 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:39.601785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:39.601888 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:39.602406 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:39.602423 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:39.605705 (+ 3282us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":236,"threads_started":1}
W20260810 00:52:39.606134 24996 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:39.597824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:53859 (local address 127.10.97.126:33165)
0810 00:52:39.597961 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:39.597965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:39.598349 (+ 384us) server_negotiation.cc:407] Connection header received
0810 00:52:39.598480 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:39.598484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:39.598535 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:39.598604 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:39.599307 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:39.600029 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:39.601290 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:39.601674 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:39.601758 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:39.605905 (+ 4147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.65:53859: BlockingRecv error: recv got EOF from 127.10.97.65:53859 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 10 00:52:40 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:40 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:40.083467 25000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:40.074870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.66:51351)
0810 00:52:40.075482 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:40.075503 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:40.075649 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:40.076060 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:40.076066 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:40.076467 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:40.076645 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:40.076654 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:40.077664 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:40.077670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:40.078591 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:40.078602 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:40.079291 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:40.079296 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:40.079463 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:40.080068 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:40.080086 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:40.083236 (+ 3150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":267,"threads_started":1}
W20260810 00:52:40.083626 24996 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:40.074887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:51351 (local address 127.10.97.126:33165)
0810 00:52:40.075043 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:40.075048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:40.075583 (+ 535us) server_negotiation.cc:407] Connection header received
0810 00:52:40.075795 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:40.075821 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:40.075904 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:40.075999 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:40.076787 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:40.077523 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:40.078735 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:40.079136 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:40.079250 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:40.083409 (+ 4159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.66:51351: BlockingRecv error: recv got EOF from 127.10.97.66:51351 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 10 00:52:40 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:40 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:40.599359 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:40.590893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.67:54435)
0810 00:52:40.591433 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:40.591454 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:40.591600 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:40.592041 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:40.592047 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:40.592530 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:40.592711 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:40.592720 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:40.593714 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:40.593717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:40.594655 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:40.594668 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:40.595359 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:40.595371 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:40.595519 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:40.596067 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:40.596084 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:40.599109 (+ 3025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":236,"threads_started":1}
W20260810 00:52:40.599547 25001 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:40.590955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:54435 (local address 127.10.97.126:33165)
0810 00:52:40.591736 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:40.591741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:40.591754 (+ 13us) server_negotiation.cc:407] Connection header received
0810 00:52:40.591795 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:40.591798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:40.591881 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:40.591965 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:40.592849 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:40.593580 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:40.594786 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:40.595217 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:40.595322 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:40.599308 (+ 3986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.67:54435: BlockingRecv error: recv got EOF from 127.10.97.67:54435 (error 108)
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":546,"threads_started":1}
Aug 10 00:52:40 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:40 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:40.615123 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:40.606778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.65:39793)
0810 00:52:40.607247 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:40.607262 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:40.607380 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:40.607675 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:40.607680 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:40.608159 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:40.608322 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:40.608331 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:40.609310 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:40.609316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:40.610237 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:40.610249 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:40.610956 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:40.610963 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:40.611105 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:40.611652 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:40.611674 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:40.614902 (+ 3228us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":216,"threads_started":1}
W20260810 00:52:40.615221 25001 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:40.606944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:39793 (local address 127.10.97.126:33165)
0810 00:52:40.607153 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:40.607158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:40.607328 (+ 170us) server_negotiation.cc:407] Connection header received
0810 00:52:40.607489 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:40.607494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:40.607549 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:40.607619 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:40.608449 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:40.609171 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:40.610366 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:40.610785 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:40.610887 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:40.615055 (+ 4168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.65:39793: BlockingRecv error: recv got EOF from 127.10.97.65:39793 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 10 00:52:41 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:41 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:41.093659 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:41.084265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.66:41795)
0810 00:52:41.084849 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:41.084866 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:41.084996 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:41.085324 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:41.085329 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:41.085767 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:41.085967 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:41.085977 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:41.087153 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:41.087159 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:41.088230 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:41.088243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:41.088990 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:41.088997 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:41.089173 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:41.089713 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:41.089730 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:41.093406 (+ 3676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":252,"threads_started":1}
W20260810 00:52:41.093818 25001 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:41.084342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:41795 (local address 127.10.97.126:33165)
0810 00:52:41.084519 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:41.084526 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:41.084941 (+ 415us) server_negotiation.cc:407] Connection header received
0810 00:52:41.085108 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:41.085113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:41.085188 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:41.085269 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:41.086105 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:41.087002 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:41.088360 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:41.088798 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:41.088924 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:41.093599 (+ 4675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.66:41795: BlockingRecv error: recv got EOF from 127.10.97.66:41795 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 10 00:52:41 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:41 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:41.609064 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:41.600180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.67:37359)
0810 00:52:41.600746 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:41.600766 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:41.600901 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:41.601244 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:41.601249 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:41.601696 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:41.601864 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:41.601873 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:41.602868 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:41.602872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:41.603795 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:41.603836 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:41.604567 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:41.604574 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:41.604727 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:41.605320 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:41.605338 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:41.608733 (+ 3395us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":260,"threads_started":1}
W20260810 00:52:41.609208 25006 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:41.600238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:37359 (local address 127.10.97.126:33165)
0810 00:52:41.600734 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:41.600740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:41.600835 (+ 95us) server_negotiation.cc:407] Connection header received
0810 00:52:41.601027 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:41.601033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:41.601099 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:41.601197 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:41.602001 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:41.602728 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:41.603950 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:41.604395 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:41.604501 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:41.608961 (+ 4460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.67:37359: BlockingRecv error: recv got EOF from 127.10.97.67:37359 (error 108)
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":300,"threads_started":1}
Aug 10 00:52:41 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:41 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:41.624097 25007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:41.615972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.65:57241)
0810 00:52:41.616422 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:41.616437 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:41.616537 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:41.616796 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:41.616798 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:41.617150 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:41.617287 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:41.617296 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:41.618200 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:41.618204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:41.619419 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:41.619435 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:41.620204 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:41.620209 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:41.620334 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:41.620876 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:41.620893 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:41.623785 (+ 2892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":230,"threads_started":1}
W20260810 00:52:41.624260 25006 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:41.615987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:57241 (local address 127.10.97.126:33165)
0810 00:52:41.616154 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:41.616160 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:41.616503 (+ 343us) server_negotiation.cc:407] Connection header received
0810 00:52:41.616617 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:41.616620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:41.616685 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:41.616758 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:41.617383 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:41.618104 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:41.619551 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:41.620077 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:41.620172 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:41.624026 (+ 3854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.65:57241: BlockingRecv error: recv got EOF from 127.10.97.65:57241 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 00:52:42 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:42 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:42.105571 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:42.094531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.66:48281)
0810 00:52:42.095014 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:42.095030 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:42.095148 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:42.095412 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:42.095415 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:42.095840 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:42.095996 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:42.096005 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:42.096926 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:42.096929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:42.097952 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:42.097965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:42.098694 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:42.098699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:42.098862 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:42.102097 (+ 3235us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:42.102114 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:42.105326 (+ 3212us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":255,"threads_started":1}
W20260810 00:52:42.105734 25006 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:42.094586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:48281 (local address 127.10.97.126:33165)
0810 00:52:42.094748 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:42.094753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:42.095092 (+ 339us) server_negotiation.cc:407] Connection header received
0810 00:52:42.095236 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:42.095240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:42.095311 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:42.095386 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:42.096107 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:42.096824 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:42.098083 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:42.098549 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:42.098668 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:42.105492 (+ 6824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.66:48281: BlockingRecv error: recv got EOF from 127.10.97.66:48281 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 00:52:42 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:42 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:42.618359 25009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:42.610026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.67:48253)
0810 00:52:42.610499 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:42.610515 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:42.610624 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:42.610924 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:42.610927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:42.611320 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:42.611465 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:42.611474 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:42.612442 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:42.612445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:42.613370 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:42.613381 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:42.613988 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:42.613991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:42.614107 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:42.614695 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:42.614713 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:42.618132 (+ 3419us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":227,"threads_started":1}
W20260810 00:52:42.618546 25014 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:42.610086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:48253 (local address 127.10.97.126:33165)
0810 00:52:42.610503 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:42.610507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:42.610577 (+ 70us) server_negotiation.cc:407] Connection header received
0810 00:52:42.610732 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:42.610736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:42.610806 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:42.610885 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:42.611583 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:42.612334 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:42.613467 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:42.613873 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:42.613957 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:42.618316 (+ 4359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.67:48253: BlockingRecv error: recv got EOF from 127.10.97.67:48253 (error 108)
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":220,"threads_started":1}
Aug 10 00:52:42 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:42 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:42.632881 25015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:42.624886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.65:43987)
0810 00:52:42.625335 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:42.625350 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:42.625465 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:42.625771 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:42.625776 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:42.626199 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:42.626358 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:42.626369 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:42.627529 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:42.627533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:42.628565 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:42.628578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:42.629182 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:42.629186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:42.629297 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:42.629774 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:42.629791 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:42.632661 (+ 2870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":230,"threads_started":1}
W20260810 00:52:42.632963 25014 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:42.624965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:43987 (local address 127.10.97.126:33165)
0810 00:52:42.625124 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:42.625130 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:42.625422 (+ 292us) server_negotiation.cc:407] Connection header received
0810 00:52:42.625575 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:42.625579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:42.625631 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:42.625704 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:42.626540 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:42.627405 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:42.628678 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:42.629075 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:42.629162 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:42.632810 (+ 3648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.65:43987: BlockingRecv error: recv got EOF from 127.10.97.65:43987 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 00:52:43 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:43 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:43.114718 25017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:43.106334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.66:45221)
0810 00:52:43.106809 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:43.106824 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:43.106958 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:43.107274 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:43.107279 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:43.107679 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:43.107904 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:43.107916 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:43.109147 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:43.109151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:43.110089 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:43.110100 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:43.110738 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:43.110742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:43.110847 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:43.111372 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:43.111388 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:43.114468 (+ 3080us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":269,"threads_started":1}
W20260810 00:52:43.114840 25014 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:43.106386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:45221 (local address 127.10.97.126:33165)
0810 00:52:43.106549 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:43.106555 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:43.106903 (+ 348us) server_negotiation.cc:407] Connection header received
0810 00:52:43.107067 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:43.107070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:43.107145 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:43.107225 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:43.108048 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:43.109005 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:43.110202 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:43.110608 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:43.110696 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:43.114631 (+ 3935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.66:45221: BlockingRecv error: recv got EOF from 127.10.97.66:45221 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 10 00:52:43 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:43 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:43.627449 25018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:43.619126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.67:37729)
0810 00:52:43.619606 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:43.619622 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:43.619733 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:43.620052 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:43.620056 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:43.620405 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:43.620571 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:43.620580 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:43.621511 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:43.621515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:43.622486 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:43.622496 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:43.623443 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:43.623448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:43.623573 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:43.624239 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:43.624257 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:43.627201 (+ 2944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":231,"threads_started":1}
W20260810 00:52:43.627617 25020 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:43.619258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:37729 (local address 127.10.97.126:33165)
0810 00:52:43.619659 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:43.619663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:43.619679 (+ 16us) server_negotiation.cc:407] Connection header received
0810 00:52:43.619844 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:43.619850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:43.619933 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:43.620019 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:43.620680 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:43.621404 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:43.622594 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:43.623274 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:43.623388 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:43.627407 (+ 4019us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.67:37729: BlockingRecv error: recv got EOF from 127.10.97.67:37729 (error 108)
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":214,"threads_started":1}
Aug 10 00:52:43 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:43 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:43.641793 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:43.633726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.65:39921)
0810 00:52:43.634132 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:43.634148 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:43.634251 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:43.634561 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:43.634566 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:43.635031 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:43.635208 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:43.635218 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:43.636315 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:43.636321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:43.637282 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:43.637294 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:43.637955 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:43.637961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:43.638136 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:43.638591 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:43.638606 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:43.641497 (+ 2891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":210,"threads_started":1}
W20260810 00:52:43.641877 25020 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:43.633737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:39921 (local address 127.10.97.126:33165)
0810 00:52:43.633890 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:43.633896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:43.634214 (+ 318us) server_negotiation.cc:407] Connection header received
0810 00:52:43.634357 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:43.634362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:43.634429 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:43.634523 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:43.635338 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:43.636194 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:43.637406 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:43.637817 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:43.637916 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:43.641672 (+ 3756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.65:39921: BlockingRecv error: recv got EOF from 127.10.97.65:39921 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 10 00:52:44 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:44 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:44.124095 25023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:44.115515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.66:42307)
0810 00:52:44.115997 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:44.116013 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:44.116137 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:44.116405 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:44.116408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:44.116828 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:44.117007 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:44.117015 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:44.117963 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:44.117967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:44.118888 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:44.118898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:44.119730 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:44.119734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:44.119943 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:44.120536 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:44.120555 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:44.123766 (+ 3211us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":271,"threads_started":1}
W20260810 00:52:44.124210 25020 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:44.115609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:42307 (local address 127.10.97.126:33165)
0810 00:52:44.115772 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:44.115777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:44.116077 (+ 300us) server_negotiation.cc:407] Connection header received
0810 00:52:44.116223 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:44.116227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:44.116300 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:44.116377 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:44.117108 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:44.117863 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:44.119017 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:44.119583 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:44.119681 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:44.123963 (+ 4282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.66:42307: BlockingRecv error: recv got EOF from 127.10.97.66:42307 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 00:52:44 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:44 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:44.636452 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:44.628281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.67:49713)
0810 00:52:44.628747 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:44.628763 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:44.628868 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:44.629155 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:44.629158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:44.629555 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:44.629740 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:44.629749 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:44.630674 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:44.630677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:44.631609 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:44.631621 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:44.632405 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:44.632410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:44.632540 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:44.633064 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:44.633080 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:44.636223 (+ 3143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":226,"threads_started":1}
W20260810 00:52:44.636591 25027 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:44.628346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:49713 (local address 127.10.97.126:33165)
0810 00:52:44.628804 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:44.628808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:44.628824 (+ 16us) server_negotiation.cc:407] Connection header received
0810 00:52:44.628969 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:44.628973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:44.629031 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:44.629111 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:44.629847 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:44.630574 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:44.631722 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:44.632272 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:44.632365 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:44.636388 (+ 4023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.67:49713: BlockingRecv error: recv got EOF from 127.10.97.67:49713 (error 108)
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":275,"threads_started":1}
Aug 10 00:52:44 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:44 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:44.650671 25030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:44.642611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.65:54127)
0810 00:52:44.643107 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:44.643122 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:44.643229 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:44.643474 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:44.643478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:44.643899 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:44.644076 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:44.644085 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:44.644964 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:44.644967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:44.645886 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:44.645897 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:44.646551 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:44.646555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:44.646692 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:44.647238 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:44.647256 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:44.650348 (+ 3092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":255,"threads_started":1}
W20260810 00:52:44.650774 25027 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:44.642656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:54127 (local address 127.10.97.126:33165)
0810 00:52:44.642783 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:44.642787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:44.643183 (+ 396us) server_negotiation.cc:407] Connection header received
0810 00:52:44.643318 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:44.643322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:44.643377 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:44.643442 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:44.644183 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:44.644864 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:44.645989 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:44.646418 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:44.646506 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:44.650548 (+ 4042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.65:54127: BlockingRecv error: recv got EOF from 127.10.97.65:54127 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 10 00:52:45 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:45 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:45.133430 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:45.124981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.66:43123)
0810 00:52:45.125518 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:45.125534 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:45.125656 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:45.125931 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:45.125934 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:45.126348 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:45.126518 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:45.126526 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:45.127483 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:45.127487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:45.128467 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:45.128479 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:45.129066 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:45.129070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:45.129193 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:45.129727 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:45.129743 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:45.133191 (+ 3448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":287,"threads_started":1}
W20260810 00:52:45.133572 25027 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:45.125028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:43123 (local address 127.10.97.126:33165)
0810 00:52:45.125199 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:45.125203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:45.125597 (+ 394us) server_negotiation.cc:407] Connection header received
0810 00:52:45.125744 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:45.125747 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:45.125826 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:45.125902 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:45.126623 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:45.127376 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:45.128570 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:45.128968 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:45.129052 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:45.133347 (+ 4295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.66:43123: BlockingRecv error: recv got EOF from 127.10.97.66:43123 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 00:52:45 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:45 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:45.645875 25032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:45.637304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.67:37903)
0810 00:52:45.637840 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:45.637856 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:45.637985 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:45.638285 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:45.638288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:45.638773 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:45.638943 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:45.638952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:45.639909 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:45.639912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:45.640816 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:45.640827 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:45.641474 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:45.641478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:45.641598 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:45.642139 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:45.642155 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:45.645551 (+ 3396us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":270,"threads_started":1}
W20260810 00:52:45.646019 25033 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:45.637361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:37903 (local address 127.10.97.126:33165)
0810 00:52:45.637771 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:45.637776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:45.637945 (+ 169us) server_negotiation.cc:407] Connection header received
0810 00:52:45.638083 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:45.638087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:45.638157 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:45.638241 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:45.639057 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:45.639790 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:45.640914 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:45.641345 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:45.641433 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:45.645737 (+ 4304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.67:37903: BlockingRecv error: recv got EOF from 127.10.97.67:37903 (error 108)
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":213,"threads_started":1}
Aug 10 00:52:45 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:45 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:45.659646 25036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:45.651486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.65:33273)
0810 00:52:45.651954 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:45.651968 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:45.652084 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:45.652385 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:45.652390 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:45.652775 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:45.652913 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:45.652921 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:45.653853 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:45.653857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:45.654762 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:45.654774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:45.655440 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:45.655446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:45.655600 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:45.656101 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:45.656118 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:45.659381 (+ 3263us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":226,"threads_started":1}
W20260810 00:52:45.659740 25033 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:45.651588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:33273 (local address 127.10.97.126:33165)
0810 00:52:45.651794 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:45.651817 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:45.652039 (+ 222us) server_negotiation.cc:407] Connection header received
0810 00:52:45.652198 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:45.652204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:45.652272 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:45.652343 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:45.653042 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:45.653714 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:45.654893 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:45.655306 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:45.655408 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:45.659557 (+ 4149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.65:33273: BlockingRecv error: recv got EOF from 127.10.97.65:33273 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 10 00:52:46 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:46 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:46.142583 25037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:46.134260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.66:54053)
0810 00:52:46.134685 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:46.134701 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:46.134814 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:46.135073 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:46.135076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:46.135476 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:46.135626 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:46.135635 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:46.136621 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:46.136625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:46.137530 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:46.137541 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:46.138268 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:46.138273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:46.138425 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:46.139068 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:46.139087 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:46.142343 (+ 3256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":213,"threads_started":1}
W20260810 00:52:46.142741 25033 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:46.134408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:54053 (local address 127.10.97.126:33165)
0810 00:52:46.134634 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:46.134640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:46.134763 (+ 123us) server_negotiation.cc:407] Connection header received
0810 00:52:46.134895 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:46.134899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:46.134970 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:46.135047 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:46.135730 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:46.136506 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:46.137665 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:46.138129 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:46.138228 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:46.142526 (+ 4298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.66:54053: BlockingRecv error: recv got EOF from 127.10.97.66:54053 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 10 00:52:46 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:46 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.67@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:46.655292 25038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:46.646916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.67:48547)
0810 00:52:46.647368 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:46.647388 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:46.647497 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:46.647904 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:46.647908 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:46.648312 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:46.648456 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:46.648465 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:46.649373 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:46.649377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:46.650274 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:46.650285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:46.650921 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:46.650924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:46.651039 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:46.651561 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:46.651578 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:46.654974 (+ 3396us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":226,"threads_started":1}
W20260810 00:52:46.655452 25039 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:46.646984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:48547 (local address 127.10.97.126:33165)
0810 00:52:46.647635 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:46.647640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:46.647655 (+ 15us) server_negotiation.cc:407] Connection header received
0810 00:52:46.647703 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:46.647706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:46.647762 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:46.647888 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:46.648562 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:46.649263 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:46.650378 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:46.650817 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:46.650904 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:46.655192 (+ 4288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.67:48547: BlockingRecv error: recv got EOF from 127.10.97.67:48547 (error 108)
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":446,"threads_started":1}
Aug 10 00:52:46 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:46 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.65@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:46.668672 25040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:46.660672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.65:58489)
0810 00:52:46.661104 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:46.661119 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:46.661219 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:46.661452 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:46.661455 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:46.661781 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:46.661958 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:46.661966 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:46.662863 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:46.662866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:46.663780 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:46.663791 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:46.664430 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:46.664434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:46.664557 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:46.665030 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:46.665046 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:46.668316 (+ 3270us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":206,"threads_started":1}
W20260810 00:52:46.668721 25039 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:46.660737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:58489 (local address 127.10.97.126:33165)
0810 00:52:46.660944 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:46.660949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:46.661178 (+ 229us) server_negotiation.cc:407] Connection header received
0810 00:52:46.661290 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:46.661293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:46.661343 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:46.661412 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:46.662055 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:46.662758 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:46.663923 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:46.664324 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:46.664405 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:46.668503 (+ 4098us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.65:58489: BlockingRecv error: recv got EOF from 127.10.97.65:58489 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 10 00:52:47 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 00:52:47 dist-test-slave-jlzn krb5kdc[23465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.97.66@KRBTEST.COM for oryx/127.10.97.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 00:52:47.155274 25041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:47.143727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33165 (local address 127.10.97.66:43651)
0810 00:52:47.144258 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:47.144273 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:47.144396 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:47.144724 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:47.144729 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:47.145108 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:47.145298 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:47.145308 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:47.146286 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:47.146290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:47.147385 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:47.147398 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:47.148197 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:47.148202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:47.148365 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:47.151758 (+ 3393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:47.151775 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:47.155032 (+ 3257us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.97.126:33165: Server oryx/127.10.97.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":289,"threads_started":1}
W20260810 00:52:47.155428 25039 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:47.143799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:43651 (local address 127.10.97.126:33165)
0810 00:52:47.143998 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:47.144005 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:47.144353 (+ 348us) server_negotiation.cc:407] Connection header received
0810 00:52:47.144507 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:47.144512 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:47.144585 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:47.144674 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:47.145435 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:47.146153 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:47.147508 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:47.148041 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:47.148146 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:47.155209 (+ 7063us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.97.66:43651: BlockingRecv error: recv got EOF from 127.10.97.66:43651 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 00:52:47.467994 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 24343
I20260810 00:52:47.479490 25043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:47.468991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35420 (local address 127.10.97.65:43257)
0810 00:52:47.469605 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:47.469612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:47.469645 (+ 33us) server_negotiation.cc:407] Connection header received
0810 00:52:47.469705 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:47.469711 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:47.469935 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:47.470092 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:47.471422 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:47.472531 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:47.473612 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:47.474130 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:47.474227 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:47.476444 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:47.476453 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:47.476466 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:47.476503 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:47.478993 (+ 2490us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.97.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 00:52:47.479264 (+ 271us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35420: Request ticket server kudu/127.10.97.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":323,"threads_started":1}
W20260810 00:52:47.479592 25043 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35420: Request ticket server kudu/127.10.97.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 00:52:47.479609 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 24343: Not authorized: Client connection negotiation failed: client connection to 127.10.97.65:43257: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.97.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 00:52:47.479840 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 24343
I20260810 00:52:47.502164 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 24563
I20260810 00:52:47.514313 25044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:47.503240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48972 (local address 127.10.97.66:41817)
0810 00:52:47.503901 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:47.503908 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:47.503974 (+ 66us) server_negotiation.cc:407] Connection header received
0810 00:52:47.504142 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:47.504149 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:47.504379 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:47.504540 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:47.506023 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:47.507094 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:47.508335 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:47.508957 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:47.509056 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:47.511272 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:47.511280 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:47.511292 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:47.511331 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:47.513812 (+ 2481us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.97.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 00:52:47.514059 (+ 247us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48972: Request ticket server kudu/127.10.97.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":292,"threads_started":1}
W20260810 00:52:47.514369 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 24563: Not authorized: Client connection negotiation failed: client connection to 127.10.97.66:41817: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.97.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 00:52:47.514415 25044 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48972: Request ticket server kudu/127.10.97.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 00:52:47.514529 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 24563
I20260810 00:52:47.535326 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 24767
I20260810 00:52:47.546842 25046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:47.536262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38866 (local address 127.10.97.67:37197)
0810 00:52:47.536850 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:47.536858 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:47.536888 (+ 30us) server_negotiation.cc:407] Connection header received
0810 00:52:47.536961 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:47.536970 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:47.537272 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:47.537465 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:47.538725 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:47.539919 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:47.541043 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:47.541570 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:47.541671 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:47.543979 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:47.543986 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:47.543999 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:47.544033 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:47.546293 (+ 2260us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.97.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 00:52:47.546554 (+ 261us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38866: Request ticket server kudu/127.10.97.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":249,"threads_started":1}
W20260810 00:52:47.546892 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 24767: Not authorized: Client connection negotiation failed: client connection to 127.10.97.67:37197: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.97.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 00:52:47.546938 25046 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38866: Request ticket server kudu/127.10.97.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 00:52:47.547032 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 24767
I20260810 00:52:47.572585 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 24242
I20260810 00:52:47.573809 24331 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:47.690828 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 24242
2026-08-10T00:52:47Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21193 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T00:52:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:52:47Z Disabled control of system clock
I20260810 00:52:47.776957 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:38997
--webserver_interface=127.10.97.126
--webserver_port=0
--builtin_ntp_servers=127.10.97.84:42763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:38997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 00:52:48.093968 25058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:48.094233 25058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:48.094324 25058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:48.101378 25058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:52:48.101478 25058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:48.101522 25058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:48.101577 25058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:52:48.101621 25058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:52:48.111558 25058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:42763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:38997
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:38997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:48.112811 25058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:48.114495 25058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:48.125593 25071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:48.125602 25068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:48.125805 25066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:48.126238 25058 server_base.cc:1034] running on GCE node
I20260810 00:52:48.126967 25058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:48.128249 25058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:48.129447 25058 hybrid_clock.cc:648] HybridClock initialized: now 1786323168129411 us; error 23 us; skew 500 ppm
I20260810 00:52:48.129755 25058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:52:48.131877 25058 webserver.cc:460] Webserver started at http://127.10.97.126:43259/ using document root <none> and password file <none>
I20260810 00:52:48.132498 25058 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:48.132598 25058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:48.132908 25058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:48.135694 25058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "d1e611d72f8d43889dbb3472b03fe2e7"
format_stamp: "Formatted at 2026-08-10 00:52:48 on dist-test-slave-jlzn"
server_key: "dbfac0c7d340c35e28d14c736399f6e1"
server_key_iv: "d52421df6935a66d57dd3e873fa08be8"
server_key_version: "encryptionkey@0"
I20260810 00:52:48.136480 25058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "d1e611d72f8d43889dbb3472b03fe2e7"
format_stamp: "Formatted at 2026-08-10 00:52:48 on dist-test-slave-jlzn"
server_key: "dbfac0c7d340c35e28d14c736399f6e1"
server_key_iv: "d52421df6935a66d57dd3e873fa08be8"
server_key_version: "encryptionkey@0"
I20260810 00:52:48.141129 25058 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260810 00:52:48.144227 25079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:48.145393 25058 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260810 00:52:48.145555 25058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "d1e611d72f8d43889dbb3472b03fe2e7"
format_stamp: "Formatted at 2026-08-10 00:52:48 on dist-test-slave-jlzn"
server_key: "dbfac0c7d340c35e28d14c736399f6e1"
server_key_iv: "d52421df6935a66d57dd3e873fa08be8"
server_key_version: "encryptionkey@0"
I20260810 00:52:48.145761 25058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:48.156955 25058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:48.157655 25058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:48.157913 25058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:48.174225 25058 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:38997
I20260810 00:52:48.174255 25145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:38997 every 8 connection(s)
I20260810 00:52:48.175391 25058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:52:48.175792 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 25058
I20260810 00:52:48.176014 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:52:48.176837 10629 external_mini_cluster.cc:1428] Setting key f1d0eaedf96ae97402fb665949b3dccb
I20260810 00:52:48.179939 25146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:48.191622 25151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:48.178837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42738 (local address 127.10.97.126:38997)
0810 00:52:48.179778 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:48.179792 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:48.179864 (+ 72us) server_negotiation.cc:407] Connection header received
0810 00:52:48.182039 (+ 2175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:48.182065 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:48.182585 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:48.183006 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:48.183523 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:48.185476 (+ 1953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:48.186777 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:48.187751 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:48.187968 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:48.188987 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:48.188998 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:48.189014 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:52:48.189693 (+ 679us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:48.189787 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:52:48.189826 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:48.189927 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:48.190106 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 00:52:48.190421 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":290,"threads_started":1}
I20260810 00:52:48.192878 25146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7: Bootstrap starting.
I20260810 00:52:48.195607 25146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:48.196569 25146 log.cc:824] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:48.198989 25146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7: No bootstrap required, opened a new log
I20260810 00:52:48.205683 25146 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e611d72f8d43889dbb3472b03fe2e7" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 38997 } }
I20260810 00:52:48.205988 25146 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:48.206082 25146 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1e611d72f8d43889dbb3472b03fe2e7, State: Initialized, Role: FOLLOWER
I20260810 00:52:48.206560 25146 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e611d72f8d43889dbb3472b03fe2e7" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 38997 } }
I20260810 00:52:48.206770 25146 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:52:48.206893 25146 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:52:48.207046 25146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:48.209046 25146 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e611d72f8d43889dbb3472b03fe2e7" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 38997 } }
I20260810 00:52:48.209473 25146 leader_election.cc:302] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1e611d72f8d43889dbb3472b03fe2e7; no voters:
I20260810 00:52:48.209971 25146 leader_election.cc:288] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:52:48.210078 25153 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:52:48.210729 25153 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7 [term 1 LEADER]: Becoming Leader. State: Replica: d1e611d72f8d43889dbb3472b03fe2e7, State: Running, Role: LEADER
I20260810 00:52:48.211227 25153 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e611d72f8d43889dbb3472b03fe2e7" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 38997 } }
I20260810 00:52:48.211843 25146 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:52:48.214056 25156 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1e611d72f8d43889dbb3472b03fe2e7. Latest consensus state: current_term: 1 leader_uuid: "d1e611d72f8d43889dbb3472b03fe2e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e611d72f8d43889dbb3472b03fe2e7" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 38997 } } }
I20260810 00:52:48.214244 25155 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1e611d72f8d43889dbb3472b03fe2e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e611d72f8d43889dbb3472b03fe2e7" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 38997 } } }
I20260810 00:52:48.214605 25156 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:48.214610 25155 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:48.216806 25167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:52:48.221460 25167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:52:48.230005 25167 catalog_manager.cc:1347] Generated new cluster ID: 4528cd5bef4f430e8ccb35a6356649ef
I20260810 00:52:48.230098 25167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:52:48.239135 25167 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:52:48.240839 25167 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:52:48.249089 25167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1e611d72f8d43889dbb3472b03fe2e7: Generated new TSK 0
I20260810 00:52:48.249857 25167 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:52:48.263213 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:38997
--builtin_ntp_servers=127.10.97.84:42763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 00:52:48.612092 25186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:48.612334 25186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:48.612421 25186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:48.619278 25186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:48.619364 25186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:48.619465 25186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:52:48.627946 25186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:42763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:38997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:48.629030 25186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:48.630640 25186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:48.642805 25197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:48.642880 25195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:48.642908 25199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:48.644001 25186 server_base.cc:1034] running on GCE node
I20260810 00:52:48.644582 25186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:48.645435 25186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:48.646631 25186 hybrid_clock.cc:648] HybridClock initialized: now 1786323168646574 us; error 43 us; skew 500 ppm
I20260810 00:52:48.646910 25186 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:52:48.649003 25186 webserver.cc:460] Webserver started at http://127.10.97.65:36297/ using document root <none> and password file <none>
I20260810 00:52:48.649598 25186 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:48.649713 25186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:48.650015 25186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:48.652698 25186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "b8e15e37e684464b91aea067fcc57694"
format_stamp: "Formatted at 2026-08-10 00:52:48 on dist-test-slave-jlzn"
server_key: "687632ba36a404a6bdb5b9ac67fedeb1"
server_key_iv: "7fbb74bcff59c16783f5c2361799723e"
server_key_version: "encryptionkey@0"
I20260810 00:52:48.653430 25186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "b8e15e37e684464b91aea067fcc57694"
format_stamp: "Formatted at 2026-08-10 00:52:48 on dist-test-slave-jlzn"
server_key: "687632ba36a404a6bdb5b9ac67fedeb1"
server_key_iv: "7fbb74bcff59c16783f5c2361799723e"
server_key_version: "encryptionkey@0"
I20260810 00:52:48.657998 25186 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:52:48.661927 25207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:48.663434 25186 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 00:52:48.663581 25186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "b8e15e37e684464b91aea067fcc57694"
format_stamp: "Formatted at 2026-08-10 00:52:48 on dist-test-slave-jlzn"
server_key: "687632ba36a404a6bdb5b9ac67fedeb1"
server_key_iv: "7fbb74bcff59c16783f5c2361799723e"
server_key_version: "encryptionkey@0"
I20260810 00:52:48.663767 25186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:48.694209 25186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:48.694911 25186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:48.695178 25186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:48.696060 25186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:48.697355 25186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:48.697441 25186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:48.697543 25186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:48.697614 25186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:48.726203 25186 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:37487
I20260810 00:52:48.726265 25382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:37487 every 8 connection(s)
I20260810 00:52:48.727361 25186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:52:48.732820 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 25186
I20260810 00:52:48.733009 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:52:48.733610 10629 external_mini_cluster.cc:1428] Setting key 425c18901c8e2e8c979f93864dd4f49b
I20260810 00:52:48.736172 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:38997
--builtin_ntp_servers=127.10.97.84:42763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 00:52:48.739245 25385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:48.729814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:34441 (local address 127.10.97.126:38997)
0810 00:52:48.730215 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:48.730220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:48.731363 (+ 1143us) server_negotiation.cc:407] Connection header received
0810 00:52:48.733141 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:48.733148 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:48.733235 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:48.733331 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:48.734389 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:48.735093 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:48.736685 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:48.737086 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:48.737196 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:48.738431 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:48.738436 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:48.738439 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:52:48.738456 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:48.738513 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:52:48.738525 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:48.738601 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:48.738806 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 00:52:48.738902 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":199,"threads_started":1}
I20260810 00:52:48.740134 25387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:48.730399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:38997 (local address 127.10.97.65:34441)
0810 00:52:48.731226 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:48.731260 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:48.732860 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:48.733568 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:48.733575 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:48.733654 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:48.734221 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:48.734238 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:48.735243 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:48.735247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:48.736542 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:48.736555 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:48.737295 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:48.737303 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:48.737519 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:48.738137 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:52:48.738140 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:48.738187 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:52:48.738255 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:48.738615 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:48.738627 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:48.738634 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 00:52:48.738779 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 00:52:48.739145 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":253,"threads_started":1}
I20260810 00:52:48.741556 25383 heartbeater.cc:347] Connected to a master server at 127.10.97.126:38997
I20260810 00:52:48.741814 25383 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:48.742506 25383 heartbeater.cc:511] Master 127.10.97.126:38997 requested a full tablet report, sending...
I20260810 00:52:48.744375 25103 ts_manager.cc:194] Registered new tserver with Master: b8e15e37e684464b91aea067fcc57694 (127.10.97.65:37487)
I20260810 00:52:48.745823 25103 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.97.65:34441
W20260810 00:52:49.056972 25389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:49.057224 25389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:49.057338 25389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:49.064209 25389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:49.064296 25389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:49.064437 25389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:52:49.073312 25389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:42763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:38997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:49.074786 25389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:49.076669 25389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:49.087951 25398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:49.088567 25400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:49.089490 25407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:49.089637 25389 server_base.cc:1034] running on GCE node
I20260810 00:52:49.090262 25389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:49.091081 25389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:49.092267 25389 hybrid_clock.cc:648] HybridClock initialized: now 1786323169092214 us; error 39 us; skew 500 ppm
I20260810 00:52:49.092553 25389 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:52:49.094625 25389 webserver.cc:460] Webserver started at http://127.10.97.66:41287/ using document root <none> and password file <none>
I20260810 00:52:49.095258 25389 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:49.095350 25389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:49.095655 25389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:49.098342 25389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "b6b54874163f4c0bb6c22bfd3cb8565b"
format_stamp: "Formatted at 2026-08-10 00:52:49 on dist-test-slave-jlzn"
server_key: "268663d34b1459a19ea3bd633153bbc7"
server_key_iv: "fc080dba5d7cc2a72af7607cfc02aea1"
server_key_version: "encryptionkey@0"
I20260810 00:52:49.099097 25389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "b6b54874163f4c0bb6c22bfd3cb8565b"
format_stamp: "Formatted at 2026-08-10 00:52:49 on dist-test-slave-jlzn"
server_key: "268663d34b1459a19ea3bd633153bbc7"
server_key_iv: "fc080dba5d7cc2a72af7607cfc02aea1"
server_key_version: "encryptionkey@0"
I20260810 00:52:49.103878 25389 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260810 00:52:49.106964 25417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:49.108178 25389 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:52:49.108325 25389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "b6b54874163f4c0bb6c22bfd3cb8565b"
format_stamp: "Formatted at 2026-08-10 00:52:49 on dist-test-slave-jlzn"
server_key: "268663d34b1459a19ea3bd633153bbc7"
server_key_iv: "fc080dba5d7cc2a72af7607cfc02aea1"
server_key_version: "encryptionkey@0"
I20260810 00:52:49.108510 25389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:49.125972 25389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:49.126684 25389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:49.126942 25389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:49.127769 25389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:49.129153 25389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:49.129254 25389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:49.129360 25389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:49.129429 25389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:49.164595 25389 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:44667
I20260810 00:52:49.164645 25583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:44667 every 8 connection(s)
I20260810 00:52:49.165831 25389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:52:49.171446 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 25389
I20260810 00:52:49.171633 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:52:49.172111 10629 external_mini_cluster.cc:1428] Setting key 0cac49f9613e738bb48997491b7991ed
I20260810 00:52:49.175253 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:38997
--builtin_ntp_servers=127.10.97.84:42763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 00:52:49.179141 25385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:49.168285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:51751 (local address 127.10.97.126:38997)
0810 00:52:49.168439 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:49.168443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:49.169795 (+ 1352us) server_negotiation.cc:407] Connection header received
0810 00:52:49.171487 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:49.171505 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:49.171593 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:49.171693 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:49.172793 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:49.173817 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:49.175949 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:49.176531 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:49.176626 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:49.178329 (+ 1703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:49.178337 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:49.178342 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:52:49.178363 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:49.178419 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:52:49.178435 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:49.178526 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:49.178727 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 00:52:49.178823 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 00:52:49.180239 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:49.168763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:38997 (local address 127.10.97.66:51751)
0810 00:52:49.169634 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:49.169676 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:49.171228 (+ 1552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:49.171942 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:49.171950 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:49.172031 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:49.172633 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:49.172650 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:49.173959 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:49.173963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:49.175744 (+ 1781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:49.175763 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:49.176914 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:49.176920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:49.177204 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:49.177991 (+ 787us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:52:49.177994 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:49.178040 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:52:49.178126 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:49.178536 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:49.178546 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:49.178552 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 00:52:49.178712 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 00:52:49.179054 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":298,"threads_started":1}
I20260810 00:52:49.181612 25584 heartbeater.cc:347] Connected to a master server at 127.10.97.126:38997
I20260810 00:52:49.181896 25584 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:49.182746 25584 heartbeater.cc:511] Master 127.10.97.126:38997 requested a full tablet report, sending...
I20260810 00:52:49.184216 25103 ts_manager.cc:194] Registered new tserver with Master: b6b54874163f4c0bb6c22bfd3cb8565b (127.10.97.66:44667)
I20260810 00:52:49.185299 25103 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.97.66:51751
W20260810 00:52:49.496162 25591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:49.496420 25591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:49.496548 25591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:49.503547 25591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:49.503657 25591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:49.503825 25591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:52:49.512598 25591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:42763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:38997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:49.513720 25591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:49.515383 25591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:49.528151 25599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:49.528867 25608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:49.528892 25601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:49.529496 25591 server_base.cc:1034] running on GCE node
I20260810 00:52:49.530047 25591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:49.530918 25591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:49.532114 25591 hybrid_clock.cc:648] HybridClock initialized: now 1786323169532069 us; error 35 us; skew 500 ppm
I20260810 00:52:49.532400 25591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:52:49.534503 25591 webserver.cc:460] Webserver started at http://127.10.97.67:43019/ using document root <none> and password file <none>
I20260810 00:52:49.535123 25591 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:49.535219 25591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:49.535529 25591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:49.538219 25591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "1f32876d7d0f40cead2d675808f72a0a"
format_stamp: "Formatted at 2026-08-10 00:52:49 on dist-test-slave-jlzn"
server_key: "09ebcd2f74fe8e15e3ad6b624c2a6fac"
server_key_iv: "309416b633a27c013083d48472310854"
server_key_version: "encryptionkey@0"
I20260810 00:52:49.538962 25591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "1f32876d7d0f40cead2d675808f72a0a"
format_stamp: "Formatted at 2026-08-10 00:52:49 on dist-test-slave-jlzn"
server_key: "09ebcd2f74fe8e15e3ad6b624c2a6fac"
server_key_iv: "309416b633a27c013083d48472310854"
server_key_version: "encryptionkey@0"
I20260810 00:52:49.543715 25591 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260810 00:52:49.546847 25615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:49.548059 25591 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:52:49.548205 25591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "1f32876d7d0f40cead2d675808f72a0a"
format_stamp: "Formatted at 2026-08-10 00:52:49 on dist-test-slave-jlzn"
server_key: "09ebcd2f74fe8e15e3ad6b624c2a6fac"
server_key_iv: "309416b633a27c013083d48472310854"
server_key_version: "encryptionkey@0"
I20260810 00:52:49.548393 25591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:49.560791 25591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:49.561530 25591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:49.561795 25591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:49.562635 25591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:49.563982 25591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:49.564082 25591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:49.564186 25591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:49.564256 25591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:49.593628 25591 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:45599
I20260810 00:52:49.593681 25771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:45599 every 8 connection(s)
I20260810 00:52:49.594825 25591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:52:49.602057 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 25591
I20260810 00:52:49.602246 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:52:49.602734 10629 external_mini_cluster.cc:1428] Setting key 23c1e7055ed4a43fc987414866004586
I20260810 00:52:49.607031 25385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:49.597211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:34069 (local address 127.10.97.126:38997)
0810 00:52:49.597388 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:49.597392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:49.598676 (+ 1284us) server_negotiation.cc:407] Connection header received
0810 00:52:49.600383 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:49.600387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:49.600452 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:49.600535 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:49.601465 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:49.602426 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:49.604186 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:49.604734 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:49.604831 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:49.606379 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:49.606386 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:49.606391 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:52:49.606413 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:49.606446 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:52:49.606459 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:49.606526 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:49.606727 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 00:52:49.606797 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 00:52:49.607928 25775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:49.597670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:38997 (local address 127.10.97.67:34069)
0810 00:52:49.598533 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:49.598575 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:49.600162 (+ 1587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:49.600700 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:49.600707 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:49.600794 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:49.601336 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:49.601352 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:49.602561 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:49.602565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:49.604053 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:49.604066 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:49.604952 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:49.604961 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:49.605242 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:49.605992 (+ 750us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:52:49.605995 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:49.606036 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:52:49.606106 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:49.606562 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:49.606571 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:49.606577 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 00:52:49.606719 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 00:52:49.606971 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":267,"threads_started":1}
I20260810 00:52:49.609028 25773 heartbeater.cc:347] Connected to a master server at 127.10.97.126:38997
I20260810 00:52:49.609279 25773 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:49.609944 25773 heartbeater.cc:511] Master 127.10.97.126:38997 requested a full tablet report, sending...
I20260810 00:52:49.611171 25103 ts_manager.cc:194] Registered new tserver with Master: 1f32876d7d0f40cead2d675808f72a0a (127.10.97.67:45599)
I20260810 00:52:49.612027 25103 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.97.67:34069
I20260810 00:52:49.617695 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 00:52:49.623311 25385 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:49.621923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42748 (local address 127.10.97.126:38997)
0810 00:52:49.622156 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:49.622162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:49.622471 (+ 309us) server_negotiation.cc:407] Connection header received
0810 00:52:49.622627 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:49.622630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:49.622699 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:49.622769 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:49.623088 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42748: BlockingRecv error: recv got EOF from 127.0.0.1:42748 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 00:52:49.624047 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 25186
I20260810 00:52:49.630712 25786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:49.624815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50188 (local address 127.10.97.65:37487)
0810 00:52:49.625250 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:49.625256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:49.625280 (+ 24us) server_negotiation.cc:407] Connection header received
0810 00:52:49.625331 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:49.625337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:49.625503 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:49.625612 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:49.625956 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:49.626885 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:49.628100 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:49.628616 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:49.628708 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:49.629562 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:49.629569 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:49.629580 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:52:49.630077 (+ 497us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:49.630118 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:52:49.630147 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:49.630240 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:49.630338 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 00:52:49.630437 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":203,"threads_started":1}
I20260810 00:52:49.631767 25371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:49.710917 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 25186
I20260810 00:52:49.732708 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 25389
I20260810 00:52:49.740238 25790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:49.733459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34876 (local address 127.10.97.66:44667)
0810 00:52:49.733956 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:49.733963 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:49.733990 (+ 27us) server_negotiation.cc:407] Connection header received
0810 00:52:49.734051 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:49.734058 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:49.734268 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:49.734416 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:49.734841 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:49.735832 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:49.737096 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:49.737651 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:49.737765 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:49.738719 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:49.738727 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:49.738740 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:52:49.739298 (+ 558us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:49.739344 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:52:49.739363 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:49.739474 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:49.739616 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 00:52:49.739767 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":216,"threads_started":1}
I20260810 00:52:49.741398 25572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:49.820031 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 25389
I20260810 00:52:49.844815 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 25591
I20260810 00:52:49.852958 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:49.845775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37320 (local address 127.10.97.67:45599)
0810 00:52:49.846434 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:49.846441 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:49.846473 (+ 32us) server_negotiation.cc:407] Connection header received
0810 00:52:49.846534 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:49.846540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:49.846777 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:49.846930 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:49.847394 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:49.848504 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:49.849797 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:49.850345 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:49.850484 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:49.851438 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:49.851447 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:49.851460 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:52:49.852144 (+ 684us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:49.852200 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:52:49.852223 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:49.852318 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:49.852443 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 00:52:49.852590 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":333,"threads_started":1}
I20260810 00:52:49.854183 25764 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:49.933076 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 25591
I20260810 00:52:49.952854 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 25058
I20260810 00:52:49.954008 25139 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:50.052857 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 25058
2026-08-10T00:52:50Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2355 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T00:52:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:52:50Z Disabled control of system clock
I20260810 00:52:50.131312 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:44497
--webserver_interface=127.10.97.126
--webserver_port=0
--builtin_ntp_servers=127.10.97.84:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:44497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 00:52:50.445305 25802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:50.445577 25802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:50.445708 25802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:50.452680 25802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:52:50.452781 25802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:50.452847 25802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:50.452924 25802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:52:50.452986 25802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:52:50.461845 25802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:44497
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:44497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:50.463043 25802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:50.464845 25802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:50.475852 25808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:50.475872 25813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:50.475833 25810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:50.476610 25802 server_base.cc:1034] running on GCE node
I20260810 00:52:50.477327 25802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:50.478564 25802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:50.479763 25802 hybrid_clock.cc:648] HybridClock initialized: now 1786323170479724 us; error 39 us; skew 500 ppm
I20260810 00:52:50.480103 25802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:52:50.482149 25802 webserver.cc:460] Webserver started at http://127.10.97.126:36561/ using document root <none> and password file <none>
I20260810 00:52:50.482826 25802 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:50.482925 25802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:50.483230 25802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:50.486001 25802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "239b2e6dee8a401ca4ac7bcb10bc608a"
format_stamp: "Formatted at 2026-08-10 00:52:50 on dist-test-slave-jlzn"
server_key: "5d3fdea4be2159a726e240997eb791f8"
server_key_iv: "8328cd22324c26f31e515be04e55065d"
server_key_version: "encryptionkey@0"
I20260810 00:52:50.486758 25802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "239b2e6dee8a401ca4ac7bcb10bc608a"
format_stamp: "Formatted at 2026-08-10 00:52:50 on dist-test-slave-jlzn"
server_key: "5d3fdea4be2159a726e240997eb791f8"
server_key_iv: "8328cd22324c26f31e515be04e55065d"
server_key_version: "encryptionkey@0"
I20260810 00:52:50.491473 25802 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260810 00:52:50.494637 25824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:50.495841 25802 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260810 00:52:50.495990 25802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "239b2e6dee8a401ca4ac7bcb10bc608a"
format_stamp: "Formatted at 2026-08-10 00:52:50 on dist-test-slave-jlzn"
server_key: "5d3fdea4be2159a726e240997eb791f8"
server_key_iv: "8328cd22324c26f31e515be04e55065d"
server_key_version: "encryptionkey@0"
I20260810 00:52:50.496178 25802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:50.501451 25802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:50.502135 25802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:50.502393 25802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:50.518986 25802 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:44497
I20260810 00:52:50.519025 25897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:44497 every 8 connection(s)
I20260810 00:52:50.520337 25802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:52:50.523595 25899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:50.530432 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 25802
I20260810 00:52:50.530587 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:52:50.531244 10629 external_mini_cluster.cc:1428] Setting key 7715f48e940b738d0cc86ab3549dbbd2
I20260810 00:52:50.533273 25899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a: Bootstrap starting.
I20260810 00:52:50.536192 25899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:50.537317 25899 log.cc:824] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:50.538847 25902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:50.533004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35504 (local address 127.10.97.126:44497)
0810 00:52:50.533970 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:50.533980 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:50.534017 (+ 37us) server_negotiation.cc:407] Connection header received
0810 00:52:50.535616 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:50.535636 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:50.536043 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:50.536386 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:50.537369 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:50.537375 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:50.537387 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:52:50.537401 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:50.537466 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:52:50.537486 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:50.537666 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:50.537793 (+ 127us) server_negotiation.cc:299] Negotiation successful
0810 00:52:50.538017 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":351,"threads_started":1}
I20260810 00:52:50.540078 25899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a: No bootstrap required, opened a new log
I20260810 00:52:50.547036 25899 raft_consensus.cc:348] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "239b2e6dee8a401ca4ac7bcb10bc608a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44497 } }
I20260810 00:52:50.547343 25899 raft_consensus.cc:374] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:50.547429 25899 raft_consensus.cc:729] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 239b2e6dee8a401ca4ac7bcb10bc608a, State: Initialized, Role: FOLLOWER
I20260810 00:52:50.547974 25899 consensus_queue.cc:260] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "239b2e6dee8a401ca4ac7bcb10bc608a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44497 } }
I20260810 00:52:50.548188 25899 raft_consensus.cc:388] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:52:50.548311 25899 raft_consensus.cc:482] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:52:50.548456 25899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:50.550659 25899 raft_consensus.cc:504] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "239b2e6dee8a401ca4ac7bcb10bc608a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44497 } }
I20260810 00:52:50.551095 25899 leader_election.cc:302] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 239b2e6dee8a401ca4ac7bcb10bc608a; no voters:
I20260810 00:52:50.551633 25899 leader_election.cc:288] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:52:50.551724 25905 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:52:50.552449 25905 raft_consensus.cc:686] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a [term 1 LEADER]: Becoming Leader. State: Replica: 239b2e6dee8a401ca4ac7bcb10bc608a, State: Running, Role: LEADER
I20260810 00:52:50.552906 25905 consensus_queue.cc:237] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "239b2e6dee8a401ca4ac7bcb10bc608a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44497 } }
I20260810 00:52:50.553464 25899 sys_catalog.cc:564] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:52:50.555734 25907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 239b2e6dee8a401ca4ac7bcb10bc608a. Latest consensus state: current_term: 1 leader_uuid: "239b2e6dee8a401ca4ac7bcb10bc608a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "239b2e6dee8a401ca4ac7bcb10bc608a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44497 } } }
I20260810 00:52:50.556952 25907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:50.556102 25906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "239b2e6dee8a401ca4ac7bcb10bc608a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "239b2e6dee8a401ca4ac7bcb10bc608a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44497 } } }
I20260810 00:52:50.558072 25906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:50.561645 25916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:52:50.565671 25916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:52:50.573642 25916 catalog_manager.cc:1347] Generated new cluster ID: 521cc31267224d46b51599b1ae796469
I20260810 00:52:50.573735 25916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:52:50.590994 25916 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:52:50.591343 25916 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:52:50.597977 25916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 239b2e6dee8a401ca4ac7bcb10bc608a: Generated new TSK 0
I20260810 00:52:50.598641 25916 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:52:50.605569 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:44497
--builtin_ntp_servers=127.10.97.84:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 00:52:50.957048 25929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:50.957316 25929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:50.957449 25929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:50.964829 25929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:50.964936 25929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:50.965095 25929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:52:50.973713 25929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:44497
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:50.974924 25929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:50.976677 25929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:50.988765 25936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:50.988865 25937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:50.989234 25939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:50.989761 25929 server_base.cc:1034] running on GCE node
I20260810 00:52:50.990291 25929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:50.991117 25929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:50.992304 25929 hybrid_clock.cc:648] HybridClock initialized: now 1786323170992264 us; error 32 us; skew 500 ppm
I20260810 00:52:50.992581 25929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:52:50.994987 25929 webserver.cc:460] Webserver started at http://127.10.97.65:36775/ using document root <none> and password file <none>
I20260810 00:52:50.995641 25929 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:50.995741 25929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:50.996063 25929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:50.998924 25929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "e1dae5b40a374e9ea86c2a91c1b3d0d7"
format_stamp: "Formatted at 2026-08-10 00:52:50 on dist-test-slave-jlzn"
server_key: "c112b87a911e8337e736450568de4baf"
server_key_iv: "8312d3898d17722f24bf5654793b6668"
server_key_version: "encryptionkey@0"
I20260810 00:52:50.999718 25929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "e1dae5b40a374e9ea86c2a91c1b3d0d7"
format_stamp: "Formatted at 2026-08-10 00:52:50 on dist-test-slave-jlzn"
server_key: "c112b87a911e8337e736450568de4baf"
server_key_iv: "8312d3898d17722f24bf5654793b6668"
server_key_version: "encryptionkey@0"
I20260810 00:52:51.004451 25929 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 00:52:51.007535 25948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:51.008805 25929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:52:51.008950 25929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "e1dae5b40a374e9ea86c2a91c1b3d0d7"
format_stamp: "Formatted at 2026-08-10 00:52:50 on dist-test-slave-jlzn"
server_key: "c112b87a911e8337e736450568de4baf"
server_key_iv: "8312d3898d17722f24bf5654793b6668"
server_key_version: "encryptionkey@0"
I20260810 00:52:51.009152 25929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:51.014268 25929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:51.014973 25929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:51.015216 25929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:51.016077 25929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:51.017342 25929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:51.017437 25929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:51.017542 25929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:51.017612 25929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:51.044718 25929 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:42483
I20260810 00:52:51.044780 26101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:42483 every 8 connection(s)
I20260810 00:52:51.045915 25929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:52:51.053498 26104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:51.048297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:50413 (local address 127.10.97.126:44497)
0810 00:52:51.048700 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:51.048705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:51.049721 (+ 1016us) server_negotiation.cc:407] Connection header received
0810 00:52:51.051389 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:51.051394 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:51.051457 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:51.051566 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:51.052773 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:51.052779 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:51.052784 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:52:51.052802 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:51.052845 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:52:51.052857 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:51.052964 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:51.053141 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 00:52:51.053208 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":209,"threads_started":1}
I20260810 00:52:51.054278 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:51.048754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:44497 (local address 127.10.97.65:50413)
0810 00:52:51.049570 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:51.049604 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:51.051134 (+ 1530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:51.051822 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:51.051835 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:51.051927 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:51.052494 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:52:51.052496 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:51.052531 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:52:51.052608 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:51.052945 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:51.052956 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:51.052963 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 00:52:51.053218 (+ 255us) client_negotiation.cc:240] Negotiation successful
0810 00:52:51.053440 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":245,"threads_started":1}
I20260810 00:52:51.055608 26102 heartbeater.cc:347] Connected to a master server at 127.10.97.126:44497
I20260810 00:52:51.055666 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 25929
I20260810 00:52:51.055886 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:52:51.055928 26102 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:51.056597 26102 heartbeater.cc:511] Master 127.10.97.126:44497 requested a full tablet report, sending...
I20260810 00:52:51.056695 10629 external_mini_cluster.cc:1428] Setting key eb389250bb34a91dcd1c6f2f42f46185
I20260810 00:52:51.058789 25845 ts_manager.cc:194] Registered new tserver with Master: e1dae5b40a374e9ea86c2a91c1b3d0d7 (127.10.97.65:42483)
I20260810 00:52:51.059386 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:44497
--builtin_ntp_servers=127.10.97.84:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 00:52:51.375084 26107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:51.375337 26107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:51.375430 26107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:51.382236 26107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:51.382324 26107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:51.382436 26107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:52:51.390904 26107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:44497
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:51.392091 26107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:51.393731 26107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:51.405802 26114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:51.405716 26116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:51.405762 26120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:51.406725 26107 server_base.cc:1034] running on GCE node
I20260810 00:52:51.407301 26107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:51.408169 26107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:51.409359 26107 hybrid_clock.cc:648] HybridClock initialized: now 1786323171409309 us; error 45 us; skew 500 ppm
I20260810 00:52:51.409646 26107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:52:51.411638 26107 webserver.cc:460] Webserver started at http://127.10.97.66:40269/ using document root <none> and password file <none>
I20260810 00:52:51.412302 26107 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:51.412400 26107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:51.412699 26107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:51.415319 26107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "142bcfae9a6343ae9aab79cf80b2f71f"
format_stamp: "Formatted at 2026-08-10 00:52:51 on dist-test-slave-jlzn"
server_key: "b9e1e6c27868e66a4494ff87a99d1d85"
server_key_iv: "ce100bc83e609b8c46672003750e4b8d"
server_key_version: "encryptionkey@0"
I20260810 00:52:51.416113 26107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "142bcfae9a6343ae9aab79cf80b2f71f"
format_stamp: "Formatted at 2026-08-10 00:52:51 on dist-test-slave-jlzn"
server_key: "b9e1e6c27868e66a4494ff87a99d1d85"
server_key_iv: "ce100bc83e609b8c46672003750e4b8d"
server_key_version: "encryptionkey@0"
I20260810 00:52:51.421135 26107 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 00:52:51.424283 26126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:51.425475 26107 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260810 00:52:51.425622 26107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "142bcfae9a6343ae9aab79cf80b2f71f"
format_stamp: "Formatted at 2026-08-10 00:52:51 on dist-test-slave-jlzn"
server_key: "b9e1e6c27868e66a4494ff87a99d1d85"
server_key_iv: "ce100bc83e609b8c46672003750e4b8d"
server_key_version: "encryptionkey@0"
I20260810 00:52:51.425827 26107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:51.430891 26107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:51.431622 26107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:51.431901 26107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:51.432734 26107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:51.434005 26107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:51.434098 26107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:51.434207 26107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:51.434278 26107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:51.461992 26107 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:43571
I20260810 00:52:51.462051 26285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:43571 every 8 connection(s)
I20260810 00:52:51.463174 26107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:52:51.470638 26104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:51.465503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:45755 (local address 127.10.97.126:44497)
0810 00:52:51.465684 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:51.465688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:51.466954 (+ 1266us) server_negotiation.cc:407] Connection header received
0810 00:52:51.468582 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:51.468588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:51.468654 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:51.468743 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:51.469927 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:51.469933 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:51.469938 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:52:51.469954 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:51.469987 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:52:51.469999 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:51.470116 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:51.470280 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 00:52:51.470347 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 00:52:51.471396 26288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:51.465974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:44497 (local address 127.10.97.66:45755)
0810 00:52:51.466810 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:51.466845 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:51.468335 (+ 1490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:51.468956 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:51.468964 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:51.469044 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:51.469649 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:52:51.469652 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:51.469688 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:52:51.469767 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:51.470082 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:51.470092 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:51.470100 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 00:52:51.470346 (+ 246us) client_negotiation.cc:240] Negotiation successful
0810 00:52:51.470570 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":255,"threads_started":1}
I20260810 00:52:51.472651 26286 heartbeater.cc:347] Connected to a master server at 127.10.97.126:44497
I20260810 00:52:51.472908 26286 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:51.473223 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 26107
I20260810 00:52:51.473357 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:52:51.473464 26286 heartbeater.cc:511] Master 127.10.97.126:44497 requested a full tablet report, sending...
I20260810 00:52:51.473810 10629 external_mini_cluster.cc:1428] Setting key 93cbcce85242cc406ebed5ad83b737af
I20260810 00:52:51.474867 25845 ts_manager.cc:194] Registered new tserver with Master: 142bcfae9a6343ae9aab79cf80b2f71f (127.10.97.66:43571)
I20260810 00:52:51.476397 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:44497
--builtin_ntp_servers=127.10.97.84:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 00:52:51.794412 26291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:51.794667 26291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:51.794786 26291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:51.802040 26291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:51.802138 26291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:51.802282 26291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:52:51.811080 26291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:44497
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:51.812311 26291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:51.813990 26291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:51.826218 26306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:51.826342 26301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:51.826388 26302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:51.827356 26291 server_base.cc:1034] running on GCE node
I20260810 00:52:51.827934 26291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:51.828819 26291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:51.830003 26291 hybrid_clock.cc:648] HybridClock initialized: now 1786323171829956 us; error 40 us; skew 500 ppm
I20260810 00:52:51.830268 26291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:52:51.832365 26291 webserver.cc:460] Webserver started at http://127.10.97.67:39197/ using document root <none> and password file <none>
I20260810 00:52:51.832974 26291 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:51.833057 26291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:51.833314 26291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:51.835950 26291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "4a97a6115303488d9331cbfc0e2efa8f"
format_stamp: "Formatted at 2026-08-10 00:52:51 on dist-test-slave-jlzn"
server_key: "54339c52c4c4c55c332d6c81195bce98"
server_key_iv: "61503fb0b0130857b2f5b3730c14cc4a"
server_key_version: "encryptionkey@0"
I20260810 00:52:51.836771 26291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "4a97a6115303488d9331cbfc0e2efa8f"
format_stamp: "Formatted at 2026-08-10 00:52:51 on dist-test-slave-jlzn"
server_key: "54339c52c4c4c55c332d6c81195bce98"
server_key_iv: "61503fb0b0130857b2f5b3730c14cc4a"
server_key_version: "encryptionkey@0"
I20260810 00:52:51.841423 26291 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:52:51.844575 26313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:51.845770 26291 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:52:51.845901 26291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "4a97a6115303488d9331cbfc0e2efa8f"
format_stamp: "Formatted at 2026-08-10 00:52:51 on dist-test-slave-jlzn"
server_key: "54339c52c4c4c55c332d6c81195bce98"
server_key_iv: "61503fb0b0130857b2f5b3730c14cc4a"
server_key_version: "encryptionkey@0"
I20260810 00:52:51.846067 26291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:51.851083 26291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:51.851781 26291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:51.852069 26291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:51.852878 26291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:51.854225 26291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:51.854313 26291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:51.854393 26291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:51.854467 26291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:51.883496 26291 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:42505
I20260810 00:52:51.883550 26476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:42505 every 8 connection(s)
I20260810 00:52:51.884748 26291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:52:51.890493 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 26291
I20260810 00:52:51.890673 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:52:51.891196 10629 external_mini_cluster.cc:1428] Setting key 7e19b678eeeeef76190746ab3371e4b2
I20260810 00:52:51.892422 26104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:51.887106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:41739 (local address 127.10.97.126:44497)
0810 00:52:51.887315 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:51.887320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:51.888596 (+ 1276us) server_negotiation.cc:407] Connection header received
0810 00:52:51.890162 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:51.890166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:51.890218 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:51.890304 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:51.891712 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:51.891716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:51.891720 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:52:51.891737 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:51.891779 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:52:51.891795 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:51.891934 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:51.892095 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 00:52:51.892160 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 00:52:51.893399 26479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:51.887556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:44497 (local address 127.10.97.67:41739)
0810 00:52:51.888440 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:51.888482 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:51.889939 (+ 1457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:51.890548 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:51.890559 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:51.890673 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:51.891349 (+ 676us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:52:51.891352 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:51.891400 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:52:51.891523 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:51.891901 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:51.891909 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:51.891915 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 00:52:51.892179 (+ 264us) client_negotiation.cc:240] Negotiation successful
0810 00:52:51.892400 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":281,"threads_started":1}
I20260810 00:52:51.894690 26477 heartbeater.cc:347] Connected to a master server at 127.10.97.126:44497
I20260810 00:52:51.894945 26477 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:51.895490 26477 heartbeater.cc:511] Master 127.10.97.126:44497 requested a full tablet report, sending...
I20260810 00:52:51.896788 25847 ts_manager.cc:194] Registered new tserver with Master: 4a97a6115303488d9331cbfc0e2efa8f (127.10.97.67:42505)
I20260810 00:52:51.906270 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 00:52:51.912205 26104 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:52:51.910583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50970 (local address 127.10.97.126:44497)
0810 00:52:51.910721 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:51.910726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:51.911264 (+ 538us) server_negotiation.cc:407] Connection header received
0810 00:52:51.911405 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:51.911411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:51.911468 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:51.911556 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:51.911960 (+ 404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50970: BlockingRecv error: recv got EOF from 127.0.0.1:50970 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 00:52:51.912752 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 25929
I20260810 00:52:51.916067 26486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:51.913848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48232 (local address 127.10.97.65:42483)
0810 00:52:51.914353 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:51.914359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:51.914382 (+ 23us) server_negotiation.cc:407] Connection header received
0810 00:52:51.914435 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:51.914440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:51.914600 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:51.914719 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:51.915446 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:51.915452 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:51.915461 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:52:51.915478 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:51.915515 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:52:51.915528 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:51.915623 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:51.915676 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 00:52:51.915752 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":258,"threads_started":1}
I20260810 00:52:51.917114 26096 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:51.995237 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 25929
I20260810 00:52:52.016610 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 26107
I20260810 00:52:52.019745 26490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:52.017435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53776 (local address 127.10.97.66:43571)
0810 00:52:52.017966 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:52.017972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:52.018000 (+ 28us) server_negotiation.cc:407] Connection header received
0810 00:52:52.018065 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:52.018070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:52.018289 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:52.018419 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:52.019172 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:52.019178 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:52.019189 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:52:52.019207 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:52.019245 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:52:52.019259 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:52.019364 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:52.019417 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 00:52:52.019497 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":265,"threads_started":1}
I20260810 00:52:52.020928 26280 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:52.100347 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 26107
I20260810 00:52:52.122007 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 26291
I20260810 00:52:52.125895 26492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:52.122911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57544 (local address 127.10.97.67:42505)
0810 00:52:52.123531 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:52.123539 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:52.123568 (+ 29us) server_negotiation.cc:407] Connection header received
0810 00:52:52.123630 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:52.123645 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:52.123897 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:52.124044 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:52.125164 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:52.125171 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:52.125181 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:52:52.125198 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:52.125236 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:52:52.125249 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:52.125406 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:52.125483 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 00:52:52.125584 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":294,"threads_started":1}
I20260810 00:52:52.127074 26470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:52.205751 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 26291
I20260810 00:52:52.229018 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 25802
I20260810 00:52:52.230182 25891 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:52.328392 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 25802
2026-08-10T00:52:52Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2279 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:52:52 dist-test-slave-jlzn krb5kdc[26497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:52:52 dist-test-slave-jlzn krb5kdc[26497](info): set up 2 sockets
Aug 10 00:52:52 dist-test-slave-jlzn krb5kdc[26497](info): commencing operation
krb5kdc: starting...
W20260810 00:52:53.527354 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.003s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:52:53 dist-test-slave-jlzn krb5kdc[26497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:52:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:52:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:52:53.755224 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:45963
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:44945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:45963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:54.081372 26516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:54.081774 26516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:54.081933 26516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:54.089120 26516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:52:54.089239 26516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:54.089319 26516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:54.089401 26516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:52:54.089462 26516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:52:54.099304 26516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:44945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:45963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:45963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:54.100525 26516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:54.101739 26516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:54.112601 26523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:54.112661 26522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:54.112712 26526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:54.113688 26516 server_base.cc:1034] running on GCE node
I20260810 00:52:54.114382 26516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:54.115623 26516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:54.116843 26516 hybrid_clock.cc:648] HybridClock initialized: now 1786323174116785 us; error 44 us; skew 500 ppm
I20260810 00:52:54.117153 26516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:54 dist-test-slave-jlzn krb5kdc[26497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323174, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:54.120072 26516 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:52:54.121438 26516 webserver.cc:460] Webserver started at http://127.10.97.126:36875/ using document root <none> and password file <none>
I20260810 00:52:54.122071 26516 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:54.122189 26516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:54.122466 26516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:54.125339 26516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "38a5ddeddd7241a29f5ef6a3a62e8916"
format_stamp: "Formatted at 2026-08-10 00:52:54 on dist-test-slave-jlzn"
server_key: "74ba625d9dd97bd1e7805afbf2a45dd2"
server_key_iv: "40e82e6537391ee486d0287696d55951"
server_key_version: "encryptionkey@0"
I20260810 00:52:54.126067 26516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "38a5ddeddd7241a29f5ef6a3a62e8916"
format_stamp: "Formatted at 2026-08-10 00:52:54 on dist-test-slave-jlzn"
server_key: "74ba625d9dd97bd1e7805afbf2a45dd2"
server_key_iv: "40e82e6537391ee486d0287696d55951"
server_key_version: "encryptionkey@0"
I20260810 00:52:54.130798 26516 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 00:52:54.133977 26537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:54.135195 26516 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 00:52:54.135349 26516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "38a5ddeddd7241a29f5ef6a3a62e8916"
format_stamp: "Formatted at 2026-08-10 00:52:54 on dist-test-slave-jlzn"
server_key: "74ba625d9dd97bd1e7805afbf2a45dd2"
server_key_iv: "40e82e6537391ee486d0287696d55951"
server_key_version: "encryptionkey@0"
I20260810 00:52:54.135540 26516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:54.150453 26516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:54.153913 26516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:54.154158 26516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:54.173887 26516 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:45963
I20260810 00:52:54.173920 26609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:45963 every 8 connection(s)
I20260810 00:52:54.175117 26516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:52:54.178524 26610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:54.185155 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 26516
I20260810 00:52:54.185341 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:52:54.185973 10629 external_mini_cluster.cc:1428] Setting key 5e904877b7f351fbcdaa70d1d88e77f8
I20260810 00:52:54.188498 26610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916: Bootstrap starting.
I20260810 00:52:54.191684 26610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:54.192945 26610 log.cc:824] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:54.196048 26610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916: No bootstrap required, opened a new log
Aug 10 00:52:54 dist-test-slave-jlzn krb5kdc[26497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:52:54.205592 26610 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a5ddeddd7241a29f5ef6a3a62e8916" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 45963 } }
I20260810 00:52:54.206032 26610 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:54.206153 26610 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a5ddeddd7241a29f5ef6a3a62e8916, State: Initialized, Role: FOLLOWER
I20260810 00:52:54.206813 26610 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a5ddeddd7241a29f5ef6a3a62e8916" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 45963 } }
I20260810 00:52:54.207093 26610 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:52:54.207252 26610 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:52:54.207433 26610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:54.209697 26615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:54.187763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60102 (local address 127.10.97.126:45963)
0810 00:52:54.188527 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:54.188538 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:54.188582 (+ 44us) server_negotiation.cc:407] Connection header received
0810 00:52:54.190262 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:54.190283 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:54.190619 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:54.190921 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:54.192777 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:54.194458 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:54.196242 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:54.196897 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:54.197018 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:54.201372 (+ 4354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:54.201387 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:54.201404 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:54.201457 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:54.205218 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:54.205851 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:54.205859 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:54.205871 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:54.206080 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:54.206530 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:54.206535 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:54.206537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:54.207254 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:54.207564 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:54.207946 (+ 382us) server_negotiation.cc:299] Negotiation successful
0810 00:52:54.208367 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":574,"thread_start_us":336,"threads_started":1}
I20260810 00:52:54.210489 26610 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a5ddeddd7241a29f5ef6a3a62e8916" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 45963 } }
I20260810 00:52:54.210959 26610 leader_election.cc:302] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38a5ddeddd7241a29f5ef6a3a62e8916; no voters:
I20260810 00:52:54.211608 26610 leader_election.cc:288] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:52:54.211694 26619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:52:54.212828 26619 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916 [term 1 LEADER]: Becoming Leader. State: Replica: 38a5ddeddd7241a29f5ef6a3a62e8916, State: Running, Role: LEADER
I20260810 00:52:54.213353 26619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a5ddeddd7241a29f5ef6a3a62e8916" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 45963 } }
I20260810 00:52:54.213681 26610 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:52:54.216750 26622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38a5ddeddd7241a29f5ef6a3a62e8916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a5ddeddd7241a29f5ef6a3a62e8916" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 45963 } } }
I20260810 00:52:54.216724 26623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38a5ddeddd7241a29f5ef6a3a62e8916. Latest consensus state: current_term: 1 leader_uuid: "38a5ddeddd7241a29f5ef6a3a62e8916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a5ddeddd7241a29f5ef6a3a62e8916" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 45963 } } }
I20260810 00:52:54.217118 26622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:54.217118 26623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916 [sys.catalog]: This master's current role is: LEADER
I20260810 00:52:54.219165 26630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:52:54.223788 26630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:52:54.232213 26630 catalog_manager.cc:1347] Generated new cluster ID: a83190d0bdb74784bc75fd57b7559c3d
I20260810 00:52:54.232304 26630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:52:54.243211 26630 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:52:54.244419 26630 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:52:54.251823 26630 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916: Generated new TSK 0
I20260810 00:52:54.252673 26630 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
I20260810 00:52:54.318696 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:45963
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:44945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:54.633704 26644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:54.633965 26644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:54.634097 26644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:54.641060 26644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:54.641166 26644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:54.641305 26644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:52:54.650470 26644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:44945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:45963
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:54.651610 26644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:54.652848 26644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:54.665318 26650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:54.665361 26651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:54.665364 26653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:54.665935 26644 server_base.cc:1034] running on GCE node
I20260810 00:52:54.666467 26644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:54.667352 26644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:54.668548 26644 hybrid_clock.cc:648] HybridClock initialized: now 1786323174668505 us; error 32 us; skew 500 ppm
I20260810 00:52:54.668820 26644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:54 dist-test-slave-jlzn krb5kdc[26497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323174, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:54.671765 26644 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:52:54.673053 26644 webserver.cc:460] Webserver started at http://127.10.97.65:37237/ using document root <none> and password file <none>
I20260810 00:52:54.673666 26644 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:54.673780 26644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:54.674067 26644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:54.676815 26644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "1671b679f3e045c89d9c7645f6e1bf3b"
format_stamp: "Formatted at 2026-08-10 00:52:54 on dist-test-slave-jlzn"
server_key: "45eaaa2c3df82e618ecd8d5805f82591"
server_key_iv: "e19f82b8ed458367a9361d826db0a834"
server_key_version: "encryptionkey@0"
I20260810 00:52:54.677547 26644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "1671b679f3e045c89d9c7645f6e1bf3b"
format_stamp: "Formatted at 2026-08-10 00:52:54 on dist-test-slave-jlzn"
server_key: "45eaaa2c3df82e618ecd8d5805f82591"
server_key_iv: "e19f82b8ed458367a9361d826db0a834"
server_key_version: "encryptionkey@0"
I20260810 00:52:54.682339 26644 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:52:54.685497 26665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:54.686666 26644 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:52:54.686817 26644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "1671b679f3e045c89d9c7645f6e1bf3b"
format_stamp: "Formatted at 2026-08-10 00:52:54 on dist-test-slave-jlzn"
server_key: "45eaaa2c3df82e618ecd8d5805f82591"
server_key_iv: "e19f82b8ed458367a9361d826db0a834"
server_key_version: "encryptionkey@0"
I20260810 00:52:54.686990 26644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:54.721071 26644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:54.724972 26644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:54.725229 26644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:54.726060 26644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:54.727277 26644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:54.727362 26644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:54.727484 26644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:54.727536 26644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:54.755973 26644 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:34175
I20260810 00:52:54.756016 26831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:34175 every 8 connection(s)
I20260810 00:52:54.757164 26644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:52:54.760499 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 26644
I20260810 00:52:54.760720 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:52:54.761456 10629 external_mini_cluster.cc:1428] Setting key 6fc0800617d2044ba4e7a7722fd20fbb
Aug 10 00:52:54 dist-test-slave-jlzn krb5kdc[26497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323174, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:52:54.775674 26835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:54.759481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:49341 (local address 127.10.97.126:45963)
0810 00:52:54.759906 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:54.759910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:54.760972 (+ 1062us) server_negotiation.cc:407] Connection header received
0810 00:52:54.762729 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:54.762734 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:54.762850 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:54.762945 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:54.764421 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:54.765110 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:54.766627 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:54.767021 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:54.767131 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:54.770355 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:54.770360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:54.770363 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:54.770394 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:54.772790 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:54.773460 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:54.773464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:54.773466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:54.773586 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:54.774010 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:54.774014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:54.774016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:54.774279 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:54.774414 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:54.775088 (+ 674us) server_negotiation.cc:299] Negotiation successful
0810 00:52:54.775316 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":191,"threads_started":1}
I20260810 00:52:54.776511 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:54.760033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:45963 (local address 127.10.97.65:49341)
0810 00:52:54.760816 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:54.760866 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:54.762471 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:54.763184 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:54.763191 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:54.763673 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:54.764258 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:54.764277 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:54.765263 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:54.765267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:54.766474 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:54.766487 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:54.767244 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:54.767252 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:54.767460 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:54.768092 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:54.768118 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:54.770148 (+ 2030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:54.772921 (+ 2773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:54.772929 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:54.772943 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:54.773334 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:54.773693 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:54.773698 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:54.773701 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:54.773891 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:54.774405 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:54.774412 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:54.774710 (+ 298us) client_negotiation.cc:769] Sending connection context
0810 00:52:54.774949 (+ 239us) client_negotiation.cc:240] Negotiation successful
0810 00:52:54.775416 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":309,"threads_started":1}
I20260810 00:52:54.777904 26832 heartbeater.cc:347] Connected to a master server at 127.10.97.126:45963
I20260810 00:52:54.778213 26832 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:54.779012 26832 heartbeater.cc:511] Master 127.10.97.126:45963 requested a full tablet report, sending...
I20260810 00:52:54.781157 26560 ts_manager.cc:194] Registered new tserver with Master: 1671b679f3e045c89d9c7645f6e1bf3b (127.10.97.65:34175)
I20260810 00:52:54.782580 26560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:49341
WARNING: no policy specified for kudu/127.10.97.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
I20260810 00:52:54.818847 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:45963
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:44945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:55.137573 26840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:55.137825 26840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:55.137915 26840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:55.144841 26840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:55.144934 26840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:55.145047 26840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:52:55.154284 26840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:44945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:45963
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:55.155407 26840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:55.156658 26840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:55.168804 26850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:55.168766 26847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:55.168876 26846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:55.169778 26840 server_base.cc:1034] running on GCE node
I20260810 00:52:55.170372 26840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:55.171192 26840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:55.172369 26840 hybrid_clock.cc:648] HybridClock initialized: now 1786323175172314 us; error 48 us; skew 500 ppm
I20260810 00:52:55.172652 26840 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:55 dist-test-slave-jlzn krb5kdc[26497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323175, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:55.175866 26840 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:52:55.177382 26840 webserver.cc:460] Webserver started at http://127.10.97.66:34831/ using document root <none> and password file <none>
I20260810 00:52:55.178155 26840 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:55.178299 26840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:55.178644 26840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:55.182267 26840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "82eaa96dea4a4a9e8237a6fb0a059932"
format_stamp: "Formatted at 2026-08-10 00:52:55 on dist-test-slave-jlzn"
server_key: "23e0fd991ddccbf33fd3d7369da24332"
server_key_iv: "97ed469bd9e3022a01812c4348475cf3"
server_key_version: "encryptionkey@0"
I20260810 00:52:55.183139 26840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "82eaa96dea4a4a9e8237a6fb0a059932"
format_stamp: "Formatted at 2026-08-10 00:52:55 on dist-test-slave-jlzn"
server_key: "23e0fd991ddccbf33fd3d7369da24332"
server_key_iv: "97ed469bd9e3022a01812c4348475cf3"
server_key_version: "encryptionkey@0"
I20260810 00:52:55.189133 26840 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.002s
I20260810 00:52:55.193174 26862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:55.194365 26840 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260810 00:52:55.194515 26840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "82eaa96dea4a4a9e8237a6fb0a059932"
format_stamp: "Formatted at 2026-08-10 00:52:55 on dist-test-slave-jlzn"
server_key: "23e0fd991ddccbf33fd3d7369da24332"
server_key_iv: "97ed469bd9e3022a01812c4348475cf3"
server_key_version: "encryptionkey@0"
I20260810 00:52:55.194705 26840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:55.208573 26840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:55.211999 26840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:55.212251 26840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:55.213089 26840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:55.214325 26840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:55.214402 26840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:55.214514 26840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:55.214569 26840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:55.244144 26840 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:40267
I20260810 00:52:55.244206 27009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:40267 every 8 connection(s)
I20260810 00:52:55.245307 26840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:52:55.251042 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 26840
I20260810 00:52:55.251194 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:52:55.251613 10629 external_mini_cluster.cc:1428] Setting key 09cad7b337f6e1d915f9fd1cb7886918
Aug 10 00:52:55 dist-test-slave-jlzn krb5kdc[26497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323175, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:52:55.264264 26835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:55.247596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:41579 (local address 127.10.97.126:45963)
0810 00:52:55.247830 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:55.247837 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:55.248867 (+ 1030us) server_negotiation.cc:407] Connection header received
0810 00:52:55.250520 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:55.250525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:55.250592 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:55.250679 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:55.252190 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.253069 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.254618 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.255201 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.255324 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:55.258549 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:55.258555 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:55.258560 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:55.258598 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:55.261536 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:55.262176 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:55.262181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:55.262184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:55.262335 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:55.262702 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:55.262706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:55.262709 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:55.263023 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:55.263162 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:55.263721 (+ 559us) server_negotiation.cc:299] Negotiation successful
0810 00:52:55.263953 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 00:52:55.264936 27014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:55.248049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:45963 (local address 127.10.97.66:41579)
0810 00:52:55.248714 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:55.248758 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:55.250293 (+ 1535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:55.250866 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:55.250874 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:55.251414 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:55.252022 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:55.252041 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.253213 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.253219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:55.254471 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:55.254483 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.255372 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.255377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:55.255557 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:55.256174 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:55.256195 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:55.258324 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:55.261669 (+ 3345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:55.261675 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:55.261685 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:55.262054 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:55.262428 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:55.262431 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:55.262433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:55.262601 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:55.263120 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:55.263125 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:55.263396 (+ 271us) client_negotiation.cc:769] Sending connection context
0810 00:52:55.263593 (+ 197us) client_negotiation.cc:240] Negotiation successful
0810 00:52:55.263968 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":254,"threads_started":1}
I20260810 00:52:55.266224 27011 heartbeater.cc:347] Connected to a master server at 127.10.97.126:45963
I20260810 00:52:55.266542 27011 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:55.267333 27011 heartbeater.cc:511] Master 127.10.97.126:45963 requested a full tablet report, sending...
I20260810 00:52:55.268731 26560 ts_manager.cc:194] Registered new tserver with Master: 82eaa96dea4a4a9e8237a6fb0a059932 (127.10.97.66:40267)
I20260810 00:52:55.269483 26560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:41579
WARNING: no policy specified for kudu/127.10.97.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
I20260810 00:52:55.306389 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:45963
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:44945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:52:55.617260 27017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:52:55.617569 27017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:52:55.617715 27017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:52:55.624722 27017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:52:55.624815 27017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:52:55.624958 27017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:52:55.634004 27017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:44945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:45963
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:52:55.635145 27017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:52:55.636413 27017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:52:55.648805 27025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:55.648846 27026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:52:55.648880 27031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:52:55.649672 27017 server_base.cc:1034] running on GCE node
I20260810 00:52:55.650231 27017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:52:55.651081 27017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:52:55.652285 27017 hybrid_clock.cc:648] HybridClock initialized: now 1786323175652240 us; error 30 us; skew 500 ppm
I20260810 00:52:55.652581 27017 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:52:55 dist-test-slave-jlzn krb5kdc[26497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323175, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:52:55.655524 27017 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:52:55.656854 27017 webserver.cc:460] Webserver started at http://127.10.97.67:45805/ using document root <none> and password file <none>
I20260810 00:52:55.657485 27017 fs_manager.cc:359] Metadata directory not provided
I20260810 00:52:55.657600 27017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:52:55.657877 27017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:52:55.660768 27017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "f77fa56766764ac4b18bcdf3500afb32"
format_stamp: "Formatted at 2026-08-10 00:52:55 on dist-test-slave-jlzn"
server_key: "68468c0fbf85a2680f6a98cd11f806a4"
server_key_iv: "525a0234f2e8632e5c794114a8015edd"
server_key_version: "encryptionkey@0"
I20260810 00:52:55.661487 27017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "f77fa56766764ac4b18bcdf3500afb32"
format_stamp: "Formatted at 2026-08-10 00:52:55 on dist-test-slave-jlzn"
server_key: "68468c0fbf85a2680f6a98cd11f806a4"
server_key_iv: "525a0234f2e8632e5c794114a8015edd"
server_key_version: "encryptionkey@0"
I20260810 00:52:55.666203 27017 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:52:55.669420 27039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:55.670572 27017 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:52:55.670706 27017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "f77fa56766764ac4b18bcdf3500afb32"
format_stamp: "Formatted at 2026-08-10 00:52:55 on dist-test-slave-jlzn"
server_key: "68468c0fbf85a2680f6a98cd11f806a4"
server_key_iv: "525a0234f2e8632e5c794114a8015edd"
server_key_version: "encryptionkey@0"
I20260810 00:52:55.670876 27017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:52:55.683061 27017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:52:55.686470 27017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:52:55.686724 27017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:52:55.687557 27017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:52:55.688786 27017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:52:55.688863 27017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:55.688978 27017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:52:55.689034 27017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:52:55.718611 27017 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:38749
I20260810 00:52:55.718652 27214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:38749 every 8 connection(s)
I20260810 00:52:55.719767 27017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:52:55.725904 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 27017
I20260810 00:52:55.726074 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:52:55.726475 10629 external_mini_cluster.cc:1428] Setting key 426ca62595af88422540b2e73bd22c8e
Aug 10 00:52:55 dist-test-slave-jlzn krb5kdc[26497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323175, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:52:55.737924 26835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:55.722108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:47851 (local address 127.10.97.126:45963)
0810 00:52:55.722276 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:55.722281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:55.723407 (+ 1126us) server_negotiation.cc:407] Connection header received
0810 00:52:55.725181 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:55.725186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:55.725253 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:55.725338 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:55.726813 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.727823 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.729612 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.730050 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.730177 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:55.733220 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:55.733225 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:55.733228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:55.733260 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:55.735483 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:55.736135 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:55.736139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:55.736141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:55.736269 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:55.736599 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:55.736601 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:55.736603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:55.736863 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:55.736968 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:55.737506 (+ 538us) server_negotiation.cc:299] Negotiation successful
0810 00:52:55.737673 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 00:52:55.738732 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:55.722563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:45963 (local address 127.10.97.67:47851)
0810 00:52:55.723253 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:55.723298 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:55.724926 (+ 1628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:55.725534 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:55.725541 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:55.726031 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:52:55.726639 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:55.726658 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.727946 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.727950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:55.729470 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:55.729481 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.730258 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.730263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:55.730449 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:55.730992 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:52:55.731012 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:52:55.733046 (+ 2034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:52:55.735602 (+ 2556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:55.735608 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:55.735618 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:55.736052 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:55.736349 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:52:55.736352 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:52:55.736354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:52:55.736519 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:52:55.736949 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:52:55.736954 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:52:55.737228 (+ 274us) client_negotiation.cc:769] Sending connection context
0810 00:52:55.737426 (+ 198us) client_negotiation.cc:240] Negotiation successful
0810 00:52:55.737768 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":254,"threads_started":1}
I20260810 00:52:55.740023 27215 heartbeater.cc:347] Connected to a master server at 127.10.97.126:45963
I20260810 00:52:55.740288 27215 heartbeater.cc:464] Registering TS with master...
I20260810 00:52:55.740962 27215 heartbeater.cc:511] Master 127.10.97.126:45963 requested a full tablet report, sending...
I20260810 00:52:55.742254 26560 ts_manager.cc:194] Registered new tserver with Master: f77fa56766764ac4b18bcdf3500afb32 (127.10.97.67:38749)
I20260810 00:52:55.743032 26560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:47851
I20260810 00:52:55.753252 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:52:55.769150 26835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:55.757565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60108 (local address 127.10.97.126:45963)
0810 00:52:55.757764 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:55.757771 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:55.758197 (+ 426us) server_negotiation.cc:407] Connection header received
0810 00:52:55.758362 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:55.758366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:55.758421 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:55.758493 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:55.760205 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.760896 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.762051 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.762423 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.762505 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:55.764513 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:55.764518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:55.764521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:55.764549 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:55.766932 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:55.767444 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:55.767448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:55.767450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:55.767565 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:55.767879 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:55.767882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:55.767883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:55.768166 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:55.768345 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:55.768582 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 00:52:55.768836 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 00:52:55.772691 26560 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:52:55.779408 26560 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:52:55.785715 26832 heartbeater.cc:503] Master 127.10.97.126:45963 was elected leader, sending a full tablet report...
I20260810 00:52:55.804762 27231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:55.790393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:38749 (local address 127.0.0.1:42830)
0810 00:52:55.791091 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:55.791230 (+ 139us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:55.791400 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:55.797453 (+ 6053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:55.797458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:55.797483 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:55.797929 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:55.797943 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.800372 (+ 2429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.800378 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:55.801972 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:55.801985 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.803671 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.803678 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:55.803890 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:55.803948 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 00:52:55.804080 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 00:52:55.804274 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":1,"thread_start_us":298,"threads_started":1}
I20260810 00:52:55.804863 27236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:55.791621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38534 (local address 127.10.97.65:34175)
0810 00:52:55.796799 (+ 5178us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:55.796808 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:55.796836 (+ 28us) server_negotiation.cc:407] Connection header received
0810 00:52:55.796928 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:55.796935 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:55.797165 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:55.797586 (+ 421us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:55.798187 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.799690 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.802353 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.803748 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.803955 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:55.804167 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:55.804388 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 00:52:55.804545 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5009,"thread_start_us":4844,"threads_started":1}
I20260810 00:52:55.805219 27235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:55.791035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34175 (local address 127.0.0.1:38534)
0810 00:52:55.791938 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:55.791956 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:55.792061 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:55.797406 (+ 5345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:55.797414 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:55.797446 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:55.798023 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:55.798033 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.799838 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.799843 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:55.802200 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:55.802215 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.804019 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.804026 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:55.804216 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:55.804249 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 00:52:55.804329 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 00:52:55.804474 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"mutex_wait_us":111,"thread_start_us":5879,"threads_started":1}
I20260810 00:52:55.805274 27234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:55.791445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:40267 (local address 127.0.0.1:45208)
0810 00:52:55.797160 (+ 5715us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:55.797177 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:55.797295 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:55.797836 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:55.797840 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:55.797878 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:55.798316 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:55.798330 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.800297 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.800302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:55.802440 (+ 2138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:55.802450 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.804467 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.804474 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:55.804636 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:55.804668 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 00:52:55.804775 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 00:52:55.804887 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5597,"mutex_wait_us":30,"thread_start_us":700,"threads_started":1}
I20260810 00:52:55.805724 27232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:55.790681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42830 (local address 127.10.97.67:38749)
0810 00:52:55.796888 (+ 6207us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:55.796896 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:55.796929 (+ 33us) server_negotiation.cc:407] Connection header received
0810 00:52:55.797025 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:55.797033 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:55.797263 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:55.798164 (+ 901us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:55.798408 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.800227 (+ 1819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.802131 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.803435 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.803607 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:55.803871 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:55.805225 (+ 1354us) server_negotiation.cc:299] Negotiation successful
0810 00:52:55.805379 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6054,"thread_start_us":5921,"threads_started":1}
I20260810 00:52:55.805982 27237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:55.791956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45208 (local address 127.10.97.66:40267)
0810 00:52:55.796426 (+ 4470us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:55.796435 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:55.797282 (+ 847us) server_negotiation.cc:407] Connection header received
0810 00:52:55.797422 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:55.797430 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:55.797648 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:55.797835 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:55.798531 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.800144 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.802597 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:55.804027 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:55.804154 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:55.804320 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:55.805533 (+ 1213us) server_negotiation.cc:299] Negotiation successful
0810 00:52:55.805689 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4296,"thread_start_us":4170,"threads_started":1}
I20260810 00:52:55.809222 26923 tablet_service.cc:1467] Processing CreateTablet for tablet 4db18b8a7ecb4af68b5ba075cafb51fb (DEFAULT_TABLE table=test-table [id=068594cf678a444aa26b5e4bc41ca6d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:55.809310 27114 tablet_service.cc:1467] Processing CreateTablet for tablet 4db18b8a7ecb4af68b5ba075cafb51fb (DEFAULT_TABLE table=test-table [id=068594cf678a444aa26b5e4bc41ca6d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:55.809310 26737 tablet_service.cc:1467] Processing CreateTablet for tablet 4db18b8a7ecb4af68b5ba075cafb51fb (DEFAULT_TABLE table=test-table [id=068594cf678a444aa26b5e4bc41ca6d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:52:55.810413 26923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db18b8a7ecb4af68b5ba075cafb51fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:55.810647 26737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db18b8a7ecb4af68b5ba075cafb51fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:55.810659 27114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db18b8a7ecb4af68b5ba075cafb51fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:52:55.821956 27238 tablet_bootstrap.cc:492] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932: Bootstrap starting.
I20260810 00:52:55.824384 27245 tablet_bootstrap.cc:492] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32: Bootstrap starting.
I20260810 00:52:55.825083 27244 tablet_bootstrap.cc:492] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b: Bootstrap starting.
I20260810 00:52:55.825675 27238 tablet_bootstrap.cc:654] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:55.826953 27238 log.cc:824] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:55.828227 27244 tablet_bootstrap.cc:654] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:55.828291 27245 tablet_bootstrap.cc:654] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32: Neither blocks nor log segments found. Creating new log.
I20260810 00:52:55.829380 27245 log.cc:824] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:55.829452 27244 log.cc:824] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b: Log is configured to *not* fsync() on all Append() calls
I20260810 00:52:55.830119 27238 tablet_bootstrap.cc:492] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932: No bootstrap required, opened a new log
I20260810 00:52:55.830406 27238 ts_tablet_manager.cc:1397] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260810 00:52:55.832489 27244 tablet_bootstrap.cc:492] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b: No bootstrap required, opened a new log
I20260810 00:52:55.832755 27244 ts_tablet_manager.cc:1397] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260810 00:52:55.837711 27245 tablet_bootstrap.cc:492] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32: No bootstrap required, opened a new log
I20260810 00:52:55.837939 27245 ts_tablet_manager.cc:1397] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.008s
I20260810 00:52:55.840274 27238 raft_consensus.cc:348] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f77fa56766764ac4b18bcdf3500afb32" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38749 } } peers { permanent_uuid: "1671b679f3e045c89d9c7645f6e1bf3b" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34175 } } peers { permanent_uuid: "82eaa96dea4a4a9e8237a6fb0a059932" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 40267 } }
I20260810 00:52:55.840698 27238 raft_consensus.cc:374] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:55.840803 27238 raft_consensus.cc:729] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82eaa96dea4a4a9e8237a6fb0a059932, State: Initialized, Role: FOLLOWER
I20260810 00:52:55.841395 27238 consensus_queue.cc:260] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f77fa56766764ac4b18bcdf3500afb32" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38749 } } peers { permanent_uuid: "1671b679f3e045c89d9c7645f6e1bf3b" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34175 } } peers { permanent_uuid: "82eaa96dea4a4a9e8237a6fb0a059932" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 40267 } }
I20260810 00:52:55.842478 27011 heartbeater.cc:503] Master 127.10.97.126:45963 was elected leader, sending a full tablet report...
I20260810 00:52:55.842479 27244 raft_consensus.cc:348] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f77fa56766764ac4b18bcdf3500afb32" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38749 } } peers { permanent_uuid: "1671b679f3e045c89d9c7645f6e1bf3b" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34175 } } peers { permanent_uuid: "82eaa96dea4a4a9e8237a6fb0a059932" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 40267 } }
I20260810 00:52:55.842876 27244 raft_consensus.cc:374] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:55.842978 27244 raft_consensus.cc:729] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1671b679f3e045c89d9c7645f6e1bf3b, State: Initialized, Role: FOLLOWER
I20260810 00:52:55.843207 27238 ts_tablet_manager.cc:1428] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260810 00:52:55.843623 27244 consensus_queue.cc:260] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f77fa56766764ac4b18bcdf3500afb32" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38749 } } peers { permanent_uuid: "1671b679f3e045c89d9c7645f6e1bf3b" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34175 } } peers { permanent_uuid: "82eaa96dea4a4a9e8237a6fb0a059932" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 40267 } }
I20260810 00:52:55.845275 27244 ts_tablet_manager.cc:1428] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b: Time spent starting tablet: real 0.012s user 0.006s sys 0.007s
I20260810 00:52:55.847405 27245 raft_consensus.cc:348] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f77fa56766764ac4b18bcdf3500afb32" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38749 } } peers { permanent_uuid: "1671b679f3e045c89d9c7645f6e1bf3b" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34175 } } peers { permanent_uuid: "82eaa96dea4a4a9e8237a6fb0a059932" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 40267 } }
I20260810 00:52:55.847769 27245 raft_consensus.cc:374] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:52:55.847925 27245 raft_consensus.cc:729] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f77fa56766764ac4b18bcdf3500afb32, State: Initialized, Role: FOLLOWER
I20260810 00:52:55.848551 27245 consensus_queue.cc:260] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f77fa56766764ac4b18bcdf3500afb32" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38749 } } peers { permanent_uuid: "1671b679f3e045c89d9c7645f6e1bf3b" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34175 } } peers { permanent_uuid: "82eaa96dea4a4a9e8237a6fb0a059932" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 40267 } }
I20260810 00:52:55.849551 27215 heartbeater.cc:503] Master 127.10.97.126:45963 was elected leader, sending a full tablet report...
I20260810 00:52:55.849875 27245 ts_tablet_manager.cc:1428] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32: Time spent starting tablet: real 0.012s user 0.010s sys 0.001s
W20260810 00:52:55.971369 27216 tablet.cc:2369] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:52:55.997117 27013 tablet.cc:2369] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:52:56.009300 26834 tablet.cc:2369] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:52:56.057212 27251 raft_consensus.cc:482] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:52:56.057512 27251 raft_consensus.cc:504] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f77fa56766764ac4b18bcdf3500afb32" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38749 } } peers { permanent_uuid: "1671b679f3e045c89d9c7645f6e1bf3b" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34175 } } peers { permanent_uuid: "82eaa96dea4a4a9e8237a6fb0a059932" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 40267 } }
I20260810 00:52:56.059208 27251 leader_election.cc:288] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f77fa56766764ac4b18bcdf3500afb32 (127.10.97.67:38749), 82eaa96dea4a4a9e8237a6fb0a059932 (127.10.97.66:40267)
I20260810 00:52:56.066039 27232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:56.059521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:39469 (local address 127.10.97.67:38749)
0810 00:52:56.059708 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:56.059713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:56.060311 (+ 598us) server_negotiation.cc:407] Connection header received
0810 00:52:56.060545 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:56.060551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:56.060633 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:56.060736 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:56.061385 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:56.062297 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:56.064209 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:56.065138 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:56.065256 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:56.065307 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:56.065674 (+ 367us) server_negotiation.cc:299] Negotiation successful
0810 00:52:56.065798 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 00:52:56.066058 27253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:56.059535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:38749 (local address 127.10.97.65:39469)
0810 00:52:56.060218 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:56.060237 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:56.060378 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:56.060825 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:56.060831 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:56.060858 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:56.061232 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:56.061245 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:56.062415 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:56.062421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:56.064052 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:56.064065 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:56.065334 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:56.065340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:56.065509 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:56.065543 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 00:52:56.065655 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 00:52:56.065787 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":361,"threads_started":1}
I20260810 00:52:56.066461 27237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:56.060494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:53029 (local address 127.10.97.66:40267)
0810 00:52:56.060721 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:56.060726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:56.061024 (+ 298us) server_negotiation.cc:407] Connection header received
0810 00:52:56.061158 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:56.061163 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:56.061235 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:56.061326 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:52:56.061929 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:56.062845 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:56.064691 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:56.065561 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:56.065704 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:56.065771 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:56.066112 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 00:52:56.066243 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 00:52:56.066514 27256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:56.060346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:40267 (local address 127.10.97.65:53029)
0810 00:52:56.060924 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0810 00:52:56.060936 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 00:52:56.061040 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:52:56.061366 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:52:56.061370 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:52:56.061393 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:52:56.061754 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:56.061766 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:56.062977 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:56.062980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:56.064529 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:52:56.064544 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:52:56.065769 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:52:56.065775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:52:56.065947 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:56.065976 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 00:52:56.066078 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 00:52:56.066207 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":283,"threads_started":1}
I20260810 00:52:56.067090 27151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db18b8a7ecb4af68b5ba075cafb51fb" candidate_uuid: "1671b679f3e045c89d9c7645f6e1bf3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f77fa56766764ac4b18bcdf3500afb32" is_pre_election: true
I20260810 00:52:56.067546 27151 raft_consensus.cc:2454] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1671b679f3e045c89d9c7645f6e1bf3b in term 0.
I20260810 00:52:56.068156 26667 leader_election.cc:302] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1671b679f3e045c89d9c7645f6e1bf3b, f77fa56766764ac4b18bcdf3500afb32; no voters:
I20260810 00:52:56.068081 26954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db18b8a7ecb4af68b5ba075cafb51fb" candidate_uuid: "1671b679f3e045c89d9c7645f6e1bf3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82eaa96dea4a4a9e8237a6fb0a059932" is_pre_election: true
I20260810 00:52:56.068511 27251 raft_consensus.cc:2781] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:52:56.068527 26954 raft_consensus.cc:2454] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1671b679f3e045c89d9c7645f6e1bf3b in term 0.
I20260810 00:52:56.068625 27251 raft_consensus.cc:482] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:52:56.068717 27251 raft_consensus.cc:3037] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:56.070922 27251 raft_consensus.cc:504] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f77fa56766764ac4b18bcdf3500afb32" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38749 } } peers { permanent_uuid: "1671b679f3e045c89d9c7645f6e1bf3b" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34175 } } peers { permanent_uuid: "82eaa96dea4a4a9e8237a6fb0a059932" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 40267 } }
I20260810 00:52:56.071485 27251 leader_election.cc:288] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [CANDIDATE]: Term 1 election: Requested vote from peers f77fa56766764ac4b18bcdf3500afb32 (127.10.97.67:38749), 82eaa96dea4a4a9e8237a6fb0a059932 (127.10.97.66:40267)
I20260810 00:52:56.071892 27151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db18b8a7ecb4af68b5ba075cafb51fb" candidate_uuid: "1671b679f3e045c89d9c7645f6e1bf3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f77fa56766764ac4b18bcdf3500afb32"
I20260810 00:52:56.071972 26954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db18b8a7ecb4af68b5ba075cafb51fb" candidate_uuid: "1671b679f3e045c89d9c7645f6e1bf3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82eaa96dea4a4a9e8237a6fb0a059932"
I20260810 00:52:56.072064 27151 raft_consensus.cc:3037] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:56.072157 26954 raft_consensus.cc:3037] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:52:56.074285 27151 raft_consensus.cc:2454] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1671b679f3e045c89d9c7645f6e1bf3b in term 1.
I20260810 00:52:56.074386 26954 raft_consensus.cc:2454] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1671b679f3e045c89d9c7645f6e1bf3b in term 1.
I20260810 00:52:56.074638 26667 leader_election.cc:302] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1671b679f3e045c89d9c7645f6e1bf3b, f77fa56766764ac4b18bcdf3500afb32; no voters:
I20260810 00:52:56.074910 27251 raft_consensus.cc:2781] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:52:56.075330 27251 raft_consensus.cc:686] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [term 1 LEADER]: Becoming Leader. State: Replica: 1671b679f3e045c89d9c7645f6e1bf3b, State: Running, Role: LEADER
I20260810 00:52:56.075842 27251 consensus_queue.cc:237] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f77fa56766764ac4b18bcdf3500afb32" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38749 } } peers { permanent_uuid: "1671b679f3e045c89d9c7645f6e1bf3b" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34175 } } peers { permanent_uuid: "82eaa96dea4a4a9e8237a6fb0a059932" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 40267 } }
I20260810 00:52:56.080329 26559 catalog_manager.cc:5495] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1671b679f3e045c89d9c7645f6e1bf3b (127.10.97.65). New cstate: current_term: 1 leader_uuid: "1671b679f3e045c89d9c7645f6e1bf3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f77fa56766764ac4b18bcdf3500afb32" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1671b679f3e045c89d9c7645f6e1bf3b" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82eaa96dea4a4a9e8237a6fb0a059932" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 40267 } health_report { overall_health: UNKNOWN } } }
I20260810 00:52:56.162209 27236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:56.157049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38536 (local address 127.10.97.65:34175)
0810 00:52:56.157267 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:56.157274 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:56.157351 (+ 77us) server_negotiation.cc:407] Connection header received
0810 00:52:56.157581 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:56.157585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:56.157685 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:56.157809 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:52:56.158187 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:56.158947 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:56.160406 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:56.160823 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:56.160919 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:56.161297 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:52:56.161800 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:52:56.161908 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:56.161947 (+ 39us) server_negotiation.cc:299] Negotiation successful
0810 00:52:56.162013 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 00:52:56.170171 27151 raft_consensus.cc:1261] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32 [term 1 FOLLOWER]: Refusing update from remote peer 1671b679f3e045c89d9c7645f6e1bf3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:52:56.170182 26954 raft_consensus.cc:1261] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932 [term 1 FOLLOWER]: Refusing update from remote peer 1671b679f3e045c89d9c7645f6e1bf3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:52:56.171037 27257 consensus_queue.cc:1035] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "82eaa96dea4a4a9e8237a6fb0a059932" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 40267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:52:56.171386 27251 consensus_queue.cc:1035] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f77fa56766764ac4b18bcdf3500afb32" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:52:56.187716 27261 mvcc.cc:205] Tried to move back new op lower bound from 7316779729583247360 to 7316779729209925632. Current Snapshot: MvccSnapshot[applied={T|T < 7316779729583247360}]
I20260810 00:52:56.189332 27262 mvcc.cc:205] Tried to move back new op lower bound from 7316779729583247360 to 7316779729209925632. Current Snapshot: MvccSnapshot[applied={T|T < 7316779729583247360}]
I20260810 00:52:56.202739 26559 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:52:56.202984 26559 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:52:56.206151 26559 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 38a5ddeddd7241a29f5ef6a3a62e8916: Sending DeleteTablet for 3 replicas of tablet 4db18b8a7ecb4af68b5ba075cafb51fb
I20260810 00:52:56.207273 27114 tablet_service.cc:1514] Processing DeleteTablet for tablet 4db18b8a7ecb4af68b5ba075cafb51fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:56 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:42830
I20260810 00:52:56.207281 26737 tablet_service.cc:1514] Processing DeleteTablet for tablet 4db18b8a7ecb4af68b5ba075cafb51fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:56 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:38534
I20260810 00:52:56.207851 26923 tablet_service.cc:1514] Processing DeleteTablet for tablet 4db18b8a7ecb4af68b5ba075cafb51fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:52:56 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:45208
I20260810 00:52:56.210709 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 26644
I20260810 00:52:56.220408 27278 tablet_replica.cc:330] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b: stopping tablet replica
I20260810 00:52:56.220760 27278 raft_consensus.cc:2229] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:52:56.221351 27278 raft_consensus.cc:2258] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:56.221994 27275 tablet_replica.cc:330] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32: stopping tablet replica
I20260810 00:52:56.222532 27277 tablet_replica.cc:330] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932: stopping tablet replica
I20260810 00:52:56.222867 27277 raft_consensus.cc:2229] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:52:56.223198 27275 raft_consensus.cc:2229] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:52:56.223740 27277 raft_consensus.cc:2258] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:56.224244 27275 raft_consensus.cc:2258] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:52:56.224798 27278 ts_tablet_manager.cc:1905] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:56.226437 27277 ts_tablet_manager.cc:1905] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:52:56.228221 27275 ts_tablet_manager.cc:1905] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 00:52:56 dist-test-slave-jlzn krb5kdc[26497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:52:56.232870 27278 ts_tablet_manager.cc:1918] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:52:56.233153 27278 log.cc:1196] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/4db18b8a7ecb4af68b5ba075cafb51fb
I20260810 00:52:56.233788 27278 ts_tablet_manager.cc:1939] T 4db18b8a7ecb4af68b5ba075cafb51fb P 1671b679f3e045c89d9c7645f6e1bf3b: Deleting consensus metadata
I20260810 00:52:56.234841 27277 ts_tablet_manager.cc:1918] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:52:56.236394 26541 catalog_manager.cc:4841] TS 1671b679f3e045c89d9c7645f6e1bf3b (127.10.97.65:34175): tablet 4db18b8a7ecb4af68b5ba075cafb51fb (table test-table [id=068594cf678a444aa26b5e4bc41ca6d7]) successfully deleted
I20260810 00:52:56.236879 27277 log.cc:1196] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/4db18b8a7ecb4af68b5ba075cafb51fb
I20260810 00:52:56.237449 27277 ts_tablet_manager.cc:1939] T 4db18b8a7ecb4af68b5ba075cafb51fb P 82eaa96dea4a4a9e8237a6fb0a059932: Deleting consensus metadata
I20260810 00:52:56.239245 26538 catalog_manager.cc:4841] TS 82eaa96dea4a4a9e8237a6fb0a059932 (127.10.97.66:40267): tablet 4db18b8a7ecb4af68b5ba075cafb51fb (table test-table [id=068594cf678a444aa26b5e4bc41ca6d7]) successfully deleted
I20260810 00:52:56.241083 27236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:56.211854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38548 (local address 127.10.97.65:34175)
0810 00:52:56.212032 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:56.212042 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:56.222723 (+ 10681us) server_negotiation.cc:407] Connection header received
0810 00:52:56.222914 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:56.222919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:56.222997 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:56.223104 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:56.224881 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:56.226060 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:56.227836 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:56.228513 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:56.228609 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:56.233786 (+ 5177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:56.233799 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:56.233820 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:56.233867 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:56.237711 (+ 3844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:56.238508 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:56.238516 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:56.238519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:56.238718 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:56.239233 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:56.239239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:56.239242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:56.239662 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:56.240008 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:56.240450 (+ 442us) server_negotiation.cc:299] Negotiation successful
0810 00:52:56.240696 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 00:52:56.241747 26824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:56.243891 27275 ts_tablet_manager.cc:1918] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:52:56.244372 27275 log.cc:1196] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/4db18b8a7ecb4af68b5ba075cafb51fb
I20260810 00:52:56.244815 27275 ts_tablet_manager.cc:1939] T 4db18b8a7ecb4af68b5ba075cafb51fb P f77fa56766764ac4b18bcdf3500afb32: Deleting consensus metadata
I20260810 00:52:56.246122 26538 catalog_manager.cc:4841] TS f77fa56766764ac4b18bcdf3500afb32 (127.10.97.67:38749): tablet 4db18b8a7ecb4af68b5ba075cafb51fb (table test-table [id=068594cf678a444aa26b5e4bc41ca6d7]) successfully deleted
I20260810 00:52:56.340278 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 26644
I20260810 00:52:56.364497 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 26840
Aug 10 00:52:56 dist-test-slave-jlzn krb5kdc[26497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.66@KRBTEST.COM
I20260810 00:52:56.378090 27237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:56.365166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45224 (local address 127.10.97.66:40267)
0810 00:52:56.365357 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:56.365363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:56.365384 (+ 21us) server_negotiation.cc:407] Connection header received
0810 00:52:56.365558 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:56.365562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:56.365628 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:56.365731 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:56.366904 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:56.367566 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:56.368760 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:56.369138 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:56.369231 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:56.373036 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:56.373049 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:56.373063 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:56.373094 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:56.375735 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:56.376424 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:56.376431 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:56.376433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:56.376561 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:56.376902 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:56.376904 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:56.376906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:56.377256 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:56.377474 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:56.377690 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 00:52:56.377841 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 00:52:56.378674 27004 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:56.464092 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 26840
I20260810 00:52:56.488600 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 27017
Aug 10 00:52:56 dist-test-slave-jlzn krb5kdc[26497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.67@KRBTEST.COM
I20260810 00:52:56.502009 27232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:52:56.489330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42846 (local address 127.10.97.67:38749)
0810 00:52:56.489510 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 00:52:56.489515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:52:56.489533 (+ 18us) server_negotiation.cc:407] Connection header received
0810 00:52:56.489690 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:52:56.489694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:52:56.489757 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:52:56.489837 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:52:56.491030 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:56.491750 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:56.492993 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:52:56.493357 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:52:56.493443 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:52:56.497382 (+ 3939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:52:56.497396 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:52:56.497413 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:52:56.497449 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:52:56.499875 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:56.500438 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:56.500445 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:56.500447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:56.500579 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:52:56.500838 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:52:56.500840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:52:56.500842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:52:56.501184 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:52:56.501394 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 00:52:56.501611 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 00:52:56.501773 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 00:52:56.502619 27205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:56.588955 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 27017
I20260810 00:52:56.610631 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 26516
I20260810 00:52:56.611757 26603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:52:56.718662 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 26516
2026-08-10T00:52:56Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4389 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:52:56 dist-test-slave-jlzn krb5kdc[27293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:52:56 dist-test-slave-jlzn krb5kdc[27293](info): set up 2 sockets
Aug 10 00:52:56 dist-test-slave-jlzn krb5kdc[27293](info): commencing operation
krb5kdc: starting...
W20260810 00:52:57.914834 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:52:58 dist-test-slave-jlzn krb5kdc[27293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:52:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:52:58Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.97.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.97.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.97.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/rangeradmin_127.10.97.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.97.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/rangeradmin_127.10.97.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.97.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.97.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.97.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/rangerlookup_127.10.97.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.97.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/rangerlookup_127.10.97.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.97.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.97.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.97.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/HTTP_127.10.97.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.97.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/HTTP_127.10.97.84@KRBTEST.COM.keytab.
I20260810 00:52:58.228344 10629 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskH92XUb/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/postgres -l logfile start
2026-08-10 00:53:00.421 UTC [27339] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 00:53:00.421 UTC [27339] LOG: listening on IPv4 address "127.10.97.84", port 37215
2026-08-10 00:53:00.424 UTC [27339] LOG: listening on Unix socket "/tmp/.s.PGSQL.37215"
2026-08-10 00:53:00.443 UTC [27345] LOG: database system was shut down at 2026-08-10 00:52:59 UTC
2026-08-10 00:53:00.446 UTC [27339] LOG: database system is ready to accept connections
I20260810 00:53:01.502516 10629 mini_postgres.cc:96] Postgres bound to 37215
2026-08-10 00:53:01.526 UTC [27358] FATAL: database "slave" does not exist
127.10.97.84:37215 - accepting connections
I20260810 00:53:01.527667 10629 mini_ranger.cc:161] Starting Ranger...
I20260810 00:53:01.556173 10629 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 00:53:01.941648 10629 mini_ranger.cc:87] Created ranger Postgres database
I20260810 00:53:01.941811 10629 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ranger-admin
2026-08-10 00:53:02,324 [I] DB FLAVOR :POSTGRES
2026-08-10 00:53:02,325 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 00:53:02,325 [I] Checking connection..
2026-08-10 00:53:02,325 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 00:53:02,589 [I] Checking connection passed.
2026-08-10 00:53:02,589 [I] --------- Verifying version history table ---------
2026-08-10 00:53:02,590 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 00:53:02,839 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 00:53:02,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 00:53:03,083 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 00:53:03,083 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 00:53:03,083 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 00:53:03.330 UTC [27447] WARNING: there is no transaction in progress
2026-08-10 00:53:03,339 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 00:53:03,339 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 00:53:03,575 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 00:53:03,575 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 00:53:03,575 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 00:53:03,827 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 00:53:04,081 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-jlzn.c.gcp-upstream.internal','N') ;"
2026-08-10 00:53:04,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 00:53:04,579 [I] Table x_portal_user does not exist in database ranger
2026-08-10 00:53:04,579 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 00:53:04,823 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 00:53:04,823 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 00:53:05,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 00:53:05,290 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 00:53:05,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 00:53:06.153 UTC [27617] WARNING: there is no transaction in progress
2026-08-10 00:53:06.168 UTC [27617] WARNING: there is no transaction in progress
2026-08-10 00:53:06.181 UTC [27617] WARNING: there is no transaction in progress
2026-08-10 00:53:06.194 UTC [27617] WARNING: there is no transaction in progress
2026-08-10 00:53:06.208 UTC [27617] WARNING: there is no transaction in progress
2026-08-10 00:53:06.223 UTC [27617] WARNING: there is no transaction in progress
2026-08-10 00:53:06.271 UTC [27617] WARNING: there is no transaction in progress
2026-08-10 00:53:06.277 UTC [27617] WARNING: there is no transaction in progress
2026-08-10 00:53:06.289 UTC [27617] WARNING: there is no transaction in progress
2026-08-10 00:53:06.298 UTC [27617] WARNING: there is no transaction in progress
2026-08-10 00:53:06.306 UTC [27617] WARNING: there is no transaction in progress
2026-08-10 00:53:06.318 UTC [27617] WARNING: there is no transaction in progress
2026-08-10 00:53:06.326 UTC [27617] WARNING: there is no transaction in progress
2026-08-10 00:53:06.332 UTC [27617] WARNING: there is no transaction in progress
2026-08-10 00:53:06.337 UTC [27617] WARNING: there is no transaction in progress
2026-08-10 00:53:06.343 UTC [27617] WARNING: there is no transaction in progress
2026-08-10 00:53:06,856 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 00:53:06,856 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 00:53:07,093 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 00:53:07,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 00:53:07,339 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 00:53:07,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 00:53:07,570 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 00:53:07,571 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 00:53:07,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 00:53:08,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-jlzn.c.gcp-upstream.internal';"
2026-08-10 00:53:08,282 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 00:53:08,282 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 00:53:08,510 [I] DB_PATCHES have already been applied
I20260810 00:53:08.516584 10629 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ranger-admin:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskH92XUb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 00:53:08.516698 10629 mini_ranger.cc:193] Using host: 127.10.97.84
I20260810 00:53:08.537827 10629 mini_ranger.cc:227] Ranger admin URL: http://127.10.97.84:51963
Aug 10, 2026 12:53:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 12:53:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 12:53:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 12:53:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 12:53:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 12:53:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 12:53:09 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 00:53:09.791122 10629 mini_ranger.cc:160] Time spent starting Ranger: real 8.263s user 0.005s sys 0.052s
Aug 10, 2026 12:53:29 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 12:53:29 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 10, 2026 12:53:29 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 12:53:29 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 12:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 12:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 12:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 12:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 12:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 12:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 12:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 12:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 12:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 12:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 12:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 12:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 12:53:29 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 12:53:30 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 00:53:30.948025 10629 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.97.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.97.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.97.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/rangerkms_127.10.97.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.97.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/rangerkms_127.10.97.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.97.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.97.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.97.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/HTTP_127.10.97.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.97.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/HTTP_127.10.97.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 00:53:31 dist-test-slave-jlzn krb5kdc[27293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323211, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 00:53:31.107304 10629 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 00:53:31.127104 10629 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 00:53:31.812282 10629 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 00:53:31.812405 10629 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ranger-kms
I20260810 00:53:31.812531 10629 mini_ranger_kms.cc:227] Using postgres at 127.10.97.84:37215
2026-08-10 00:53:32,282 [I] DB FLAVOR :POSTGRES
2026-08-10 00:53:32,283 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 00:53:32,283 [I] Checking connection
2026-08-10 00:53:32,283 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 00:53:32,520 [I] connection success
2026-08-10 00:53:32,520 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 00:53:32,520 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 00:53:32,520 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 00:53:32,774 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 00:53:32,774 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 00:53:32,774 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 00:53:32,774 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:37215/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 00:53:33,040 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 00:53:33.467584 10629 mini_ranger_kms.cc:310] Created kms service
I20260810 00:53:33.651965 10629 mini_ranger_kms.cc:326] Created kudu user
I20260810 00:53:33.691862 10629 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 00:53:33 dist-test-slave-jlzn krb5kdc[27293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323211, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
I20260810 00:53:34.037111 10629 mini_ranger_kms.cc:384] Added ranger policy
I20260810 00:53:34.037503 10629 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ranger-kms:/tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 00:53:34.037566 10629 mini_ranger_kms.cc:240] Using host: 127.10.97.84
I20260810 00:53:34.069208 10629 mini_ranger_kms.cc:276] Ranger KMS PID: 27883
I20260810 00:53:34.069360 10629 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.97.84:56889
Aug 10, 2026 12:53:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 12:53:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 12:53:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 12:53:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 12:53:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 12:53:35 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 00:53:36 dist-test-slave-jlzn krb5kdc[27293](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786323216, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.97.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:53:36.659070 10629 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.552s user 0.107s sys 0.396s
I20260810 00:53:36.659444 10629 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 00:53:36.659693 10629 mini_ranger_kms.cc:401] 127.10.97.84:56889/kms/v1/keys
Aug 10 00:53:36 dist-test-slave-jlzn krb5kdc[27293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323216, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.97.84@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
Aug 10 00:53:37 dist-test-slave-jlzn krb5kdc[27293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323216, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.97.84@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
Aug 10 00:53:39 dist-test-slave-jlzn krb5kdc[27293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:53:39.253547 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:42413
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:42481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:42413
--ranger_config_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.97.84:56889/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:53:39.592028 27962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:53:39.592278 27962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:53:39.592370 27962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:53:39.599288 27962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:53:39.599380 27962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:53:39.599431 27962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:53:39.599486 27962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:53:39.599536 27962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:53:39.609277 27962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:42481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.97.84:56889/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:42413
--ranger_config_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:42413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:53:39.610378 27962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:53:39.611647 27962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:53:39.622637 27968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:53:39.622613 27969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:53:39.622757 27971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:53:39.623648 27962 server_base.cc:1034] running on GCE node
I20260810 00:53:39.624490 27962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:53:39.625725 27962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:53:39.626912 27962 hybrid_clock.cc:648] HybridClock initialized: now 1786323219626880 us; error 28 us; skew 500 ppm
I20260810 00:53:39.627224 27962 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:53:39 dist-test-slave-jlzn krb5kdc[27293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323219, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:53:39.632758 27962 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:53:39.634068 27962 webserver.cc:460] Webserver started at http://127.10.97.126:46307/ using document root <none> and password file <none>
I20260810 00:53:39.634640 27962 fs_manager.cc:359] Metadata directory not provided
I20260810 00:53:39.634752 27962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:53:39.635026 27962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 00:53:39 dist-test-slave-jlzn krb5kdc[27293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323219, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
I20260810 00:53:39.691391 27962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "c12204e4cf59472e8577b0208224c8c9"
format_stamp: "Formatted at 2026-08-10 00:53:39 on dist-test-slave-jlzn"
server_key: "b6e6ed0b0c55a66a5a3dcbcc30076513"
server_key_iv: "09ef8d09b046aea97c5b7168d07dafb5"
server_key_version: "kuduclusterkey@0"
I20260810 00:53:39.692260 27962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "c12204e4cf59472e8577b0208224c8c9"
format_stamp: "Formatted at 2026-08-10 00:53:39 on dist-test-slave-jlzn"
server_key: "b6e6ed0b0c55a66a5a3dcbcc30076513"
server_key_iv: "09ef8d09b046aea97c5b7168d07dafb5"
server_key_version: "kuduclusterkey@0"
I20260810 00:53:39.697505 27962 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 00:53:39.712299 27980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:53:39.713721 27962 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 00:53:39.713881 27962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "c12204e4cf59472e8577b0208224c8c9"
format_stamp: "Formatted at 2026-08-10 00:53:39 on dist-test-slave-jlzn"
server_key: "b6e6ed0b0c55a66a5a3dcbcc30076513"
server_key_iv: "09ef8d09b046aea97c5b7168d07dafb5"
server_key_version: "kuduclusterkey@0"
I20260810 00:53:39.714069 27962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:53:39.730082 27962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:53:39.733508 27962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:53:39.733815 27962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:53:39.750526 27962 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:42413
I20260810 00:53:39.750566 28066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:42413 every 8 connection(s)
I20260810 00:53:39.751742 27962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:53:39.755203 28068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:53:39.758834 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 27962
I20260810 00:53:39.759037 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:53:39.759860 10629 external_mini_cluster.cc:1428] Setting key 9cccc721267f8c407017e1e61a2d4f39
I20260810 00:53:39.767858 28068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9: Bootstrap starting.
I20260810 00:53:39.771322 28068 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9: Neither blocks nor log segments found. Creating new log.
I20260810 00:53:39.772575 28068 log.cc:824] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9: Log is configured to *not* fsync() on all Append() calls
Aug 10 00:53:39 dist-test-slave-jlzn krb5kdc[27293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:53:39.775514 28068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9: No bootstrap required, opened a new log
I20260810 00:53:39.782387 28073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:39.761821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38072 (local address 127.10.97.126:42413)
0810 00:53:39.762535 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0810 00:53:39.762545 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:53:39.762583 (+ 38us) server_negotiation.cc:407] Connection header received
0810 00:53:39.764120 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:53:39.764142 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:53:39.764493 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:53:39.764823 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:53:39.766343 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:39.767598 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:39.769314 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:39.770024 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:39.770143 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:39.774395 (+ 4252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:53:39.774406 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:53:39.774424 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:53:39.774467 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:53:39.778152 (+ 3685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:53:39.778759 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:53:39.778772 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:53:39.778780 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:53:39.778973 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:53:39.779354 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:53:39.779359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:53:39.779363 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:53:39.780092 (+ 729us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:53:39.780375 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0810 00:53:39.780730 (+ 355us) server_negotiation.cc:299] Negotiation successful
0810 00:53:39.781108 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":340,"threads_started":1}
I20260810 00:53:39.785075 28068 raft_consensus.cc:348] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c12204e4cf59472e8577b0208224c8c9" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42413 } }
I20260810 00:53:39.785408 28068 raft_consensus.cc:374] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:53:39.785480 28068 raft_consensus.cc:729] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c12204e4cf59472e8577b0208224c8c9, State: Initialized, Role: FOLLOWER
I20260810 00:53:39.785938 28068 consensus_queue.cc:260] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c12204e4cf59472e8577b0208224c8c9" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42413 } }
I20260810 00:53:39.786139 28068 raft_consensus.cc:388] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:53:39.786243 28068 raft_consensus.cc:482] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:53:39.786355 28068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:53:39.788400 28068 raft_consensus.cc:504] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c12204e4cf59472e8577b0208224c8c9" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42413 } }
I20260810 00:53:39.788885 28068 leader_election.cc:302] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c12204e4cf59472e8577b0208224c8c9; no voters:
I20260810 00:53:39.789397 28068 leader_election.cc:288] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:53:39.789515 28075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:53:39.790136 28075 raft_consensus.cc:686] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9 [term 1 LEADER]: Becoming Leader. State: Replica: c12204e4cf59472e8577b0208224c8c9, State: Running, Role: LEADER
I20260810 00:53:39.790642 28075 consensus_queue.cc:237] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c12204e4cf59472e8577b0208224c8c9" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42413 } }
I20260810 00:53:39.791208 28068 sys_catalog.cc:564] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:53:39.793458 28077 sys_catalog.cc:455] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c12204e4cf59472e8577b0208224c8c9. Latest consensus state: current_term: 1 leader_uuid: "c12204e4cf59472e8577b0208224c8c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c12204e4cf59472e8577b0208224c8c9" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42413 } } }
I20260810 00:53:39.793490 28076 sys_catalog.cc:455] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c12204e4cf59472e8577b0208224c8c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c12204e4cf59472e8577b0208224c8c9" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42413 } } }
I20260810 00:53:39.793865 28076 sys_catalog.cc:458] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9 [sys.catalog]: This master's current role is: LEADER
I20260810 00:53:39.793865 28077 sys_catalog.cc:458] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9 [sys.catalog]: This master's current role is: LEADER
I20260810 00:53:39.795333 28085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:53:39.799630 28085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:53:39.807687 28085 catalog_manager.cc:1347] Generated new cluster ID: d413218648f74b8ca7314c80fc78d0a3
I20260810 00:53:39.807787 28085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:53:39.812148 28068 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 00:53:39.829043 28085 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:53:39.830876 28085 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:53:39.839221 28085 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9: Generated new TSK 0
I20260810 00:53:39.840236 28085 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
I20260810 00:53:40.607399 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.97.84:56889/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:42413
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:42481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 00:53:40 dist-test-slave-jlzn krb5kdc[27293](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786323220, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 00:53:41.037671 28151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:53:41.038035 28151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:53:41.038249 28151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:53:41.049592 28151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:53:41.049774 28151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:53:41.050000 28151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:53:41.060602 28151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:42481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.97.84:56889/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:42413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:53:41.061816 28151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:53:41.063066 28151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:53:41.075994 28173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:53:41.075964 28169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:53:41.076610 28168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:53:41.077872 28151 server_base.cc:1034] running on GCE node
I20260810 00:53:41.078464 28151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:53:41.088157 28151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:53:41.089443 28151 hybrid_clock.cc:648] HybridClock initialized: now 1786323221089359 us; error 77 us; skew 500 ppm
I20260810 00:53:41.089756 28151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:53:41 dist-test-slave-jlzn krb5kdc[27293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323221, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:53:41.093076 28151 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:53:41.097374 28151 webserver.cc:460] Webserver started at http://127.10.97.65:35563/ using document root <none> and password file <none>
I20260810 00:53:41.098088 28151 fs_manager.cc:359] Metadata directory not provided
I20260810 00:53:41.098210 28151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:53:41.098517 28151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 00:53:41 dist-test-slave-jlzn krb5kdc[27293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323221, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
I20260810 00:53:41.125452 28151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "98c2d9e1b3d2435d9348aca755a877f9"
format_stamp: "Formatted at 2026-08-10 00:53:41 on dist-test-slave-jlzn"
server_key: "d0082efc520c72bfdd96615aef555408"
server_key_iv: "11f0e7eed60df291a7bab0926e81f5fa"
server_key_version: "kuduclusterkey@0"
I20260810 00:53:41.126997 28151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "98c2d9e1b3d2435d9348aca755a877f9"
format_stamp: "Formatted at 2026-08-10 00:53:41 on dist-test-slave-jlzn"
server_key: "d0082efc520c72bfdd96615aef555408"
server_key_iv: "11f0e7eed60df291a7bab0926e81f5fa"
server_key_version: "kuduclusterkey@0"
I20260810 00:53:41.139350 28151 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.004s sys 0.007s
I20260810 00:53:41.164371 28185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:53:41.165876 28151 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.002s
I20260810 00:53:41.166056 28151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "98c2d9e1b3d2435d9348aca755a877f9"
format_stamp: "Formatted at 2026-08-10 00:53:41 on dist-test-slave-jlzn"
server_key: "d0082efc520c72bfdd96615aef555408"
server_key_iv: "11f0e7eed60df291a7bab0926e81f5fa"
server_key_version: "kuduclusterkey@0"
I20260810 00:53:41.166268 28151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:53:41.201866 28151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:53:41.205822 28151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:53:41.206207 28151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:53:41.207190 28151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:53:41.208673 28151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:53:41.208751 28151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:53:41.208873 28151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:53:41.208930 28151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 00:53:41 dist-test-slave-jlzn krb5kdc[27293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323220, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
I20260810 00:53:41.394196 28151 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:38641
I20260810 00:53:41.394245 28351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:38641 every 8 connection(s)
I20260810 00:53:41.395915 28151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:53:41.405354 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 28151
I20260810 00:53:41.405555 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:53:41.406374 10629 external_mini_cluster.cc:1428] Setting key fa2204d678265895f7bc4b70c57f7e22
Aug 10 00:53:41 dist-test-slave-jlzn krb5kdc[27293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323221, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.97.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.66@KRBTEST.COM" created.
I20260810 00:53:41.452144 28354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:41.398947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:45459 (local address 127.10.97.126:42413)
0810 00:53:41.432910 (+ 33963us) server_negotiation.cc:206] Beginning negotiation
0810 00:53:41.432916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:53:41.432937 (+ 21us) server_negotiation.cc:407] Connection header received
0810 00:53:41.432996 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:53:41.433002 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:53:41.433101 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:53:41.433205 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:53:41.435303 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:41.436500 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:41.438870 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:41.439507 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:41.439617 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:41.444927 (+ 5310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:53:41.444935 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:53:41.444939 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:53:41.444981 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:53:41.448537 (+ 3556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:53:41.449405 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:53:41.449411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:53:41.449416 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:53:41.449579 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:53:41.450080 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:53:41.450086 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:53:41.450089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:53:41.450426 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:53:41.450551 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 00:53:41.451478 (+ 927us) server_negotiation.cc:299] Negotiation successful
0810 00:53:41.451730 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33836,"thread_start_us":33727,"threads_started":1}
I20260810 00:53:41.454896 28355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:41.399484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42413 (local address 127.10.97.65:45459)
0810 00:53:41.422785 (+ 23301us) negotiation.cc:107] Waiting for socket to connect
0810 00:53:41.422847 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 00:53:41.425186 (+ 2339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:53:41.433641 (+ 8455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:53:41.433653 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:53:41.434288 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:53:41.435096 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:53:41.435122 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:53:41.436683 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:53:41.436689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:53:41.438692 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:53:41.438708 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:53:41.439934 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:53:41.439942 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:53:41.440239 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:41.441778 (+ 1539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:53:41.441811 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:53:41.444665 (+ 2854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:53:41.448695 (+ 4030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:53:41.448704 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:53:41.448720 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:53:41.449269 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:53:41.449720 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:53:41.449724 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:53:41.449728 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:53:41.449953 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:53:41.450655 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:53:41.450662 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:53:41.451054 (+ 392us) client_negotiation.cc:769] Sending connection context
0810 00:53:41.452799 (+ 1745us) client_negotiation.cc:240] Negotiation successful
0810 00:53:41.453365 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22970,"thread_start_us":32193,"threads_started":1}
I20260810 00:53:41.457000 28352 heartbeater.cc:347] Connected to a master server at 127.10.97.126:42413
I20260810 00:53:41.457350 28352 heartbeater.cc:464] Registering TS with master...
I20260810 00:53:41.458250 28352 heartbeater.cc:511] Master 127.10.97.126:42413 requested a full tablet report, sending...
I20260810 00:53:41.460788 28006 ts_manager.cc:194] Registered new tserver with Master: 98c2d9e1b3d2435d9348aca755a877f9 (127.10.97.65:38641)
I20260810 00:53:41.462847 28006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:45459
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
I20260810 00:53:41.487006 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.97.84:56889/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:42413
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:42481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 00:53:41 dist-test-slave-jlzn krb5kdc[27293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323220, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
W20260810 00:53:41.905479 28358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:53:41.905735 28358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:53:41.905845 28358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:53:41.913015 28358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:53:41.913128 28358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:53:41.913275 28358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:53:41.922948 28358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:42481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.97.84:56889/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:42413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:53:41.924185 28358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:53:41.925413 28358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:53:41.939390 28381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:53:41.940107 28383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:53:41.940145 28380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:53:41.941017 28358 server_base.cc:1034] running on GCE node
I20260810 00:53:41.941619 28358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:53:41.942507 28358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:53:41.943707 28358 hybrid_clock.cc:648] HybridClock initialized: now 1786323221943651 us; error 53 us; skew 500 ppm
I20260810 00:53:41.944020 28358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:53:41 dist-test-slave-jlzn krb5kdc[27293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323221, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:53:41.947067 28358 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:53:41.948477 28358 webserver.cc:460] Webserver started at http://127.10.97.66:41533/ using document root <none> and password file <none>
I20260810 00:53:41.949112 28358 fs_manager.cc:359] Metadata directory not provided
I20260810 00:53:41.949226 28358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:53:41.949509 28358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 00:53:41 dist-test-slave-jlzn krb5kdc[27293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323221, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
I20260810 00:53:41.967466 28358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "38476c2a69db47a7b82d08d3d018aa7d"
format_stamp: "Formatted at 2026-08-10 00:53:41 on dist-test-slave-jlzn"
server_key: "778b8e6cf96347c732021ffa5806d4a7"
server_key_iv: "e68619fbeaed3e36ff49998f5b7b47fd"
server_key_version: "kuduclusterkey@0"
I20260810 00:53:41.968575 28358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "38476c2a69db47a7b82d08d3d018aa7d"
format_stamp: "Formatted at 2026-08-10 00:53:41 on dist-test-slave-jlzn"
server_key: "778b8e6cf96347c732021ffa5806d4a7"
server_key_iv: "e68619fbeaed3e36ff49998f5b7b47fd"
server_key_version: "kuduclusterkey@0"
I20260810 00:53:41.975579 28358 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260810 00:53:41.992204 28392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:53:41.993584 28358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260810 00:53:41.993741 28358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "38476c2a69db47a7b82d08d3d018aa7d"
format_stamp: "Formatted at 2026-08-10 00:53:41 on dist-test-slave-jlzn"
server_key: "778b8e6cf96347c732021ffa5806d4a7"
server_key_iv: "e68619fbeaed3e36ff49998f5b7b47fd"
server_key_version: "kuduclusterkey@0"
I20260810 00:53:41.993943 28358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:53:42.007675 28358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:53:42.011415 28358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:53:42.011698 28358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:53:42.012647 28358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:53:42.013988 28358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:53:42.014060 28358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:53:42.014183 28358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:53:42.014233 28358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:53:42.051513 28358 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:36081
I20260810 00:53:42.051594 28570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:36081 every 8 connection(s)
I20260810 00:53:42.052788 28358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:53:42.060858 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 28358
I20260810 00:53:42.061065 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:53:42.061605 10629 external_mini_cluster.cc:1428] Setting key 5da1a446d3496ded182835d0722cfe8d
Aug 10 00:53:42 dist-test-slave-jlzn krb5kdc[27293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323221, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:53:42.072571 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:42.055227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:53705 (local address 127.10.97.126:42413)
0810 00:53:42.055631 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0810 00:53:42.055636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:53:42.056514 (+ 878us) server_negotiation.cc:407] Connection header received
0810 00:53:42.058254 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:53:42.058261 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:53:42.058344 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:53:42.058441 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:53:42.059858 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.060601 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.062726 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.063334 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.063455 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:42.066992 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:53:42.066997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:53:42.067000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:53:42.067033 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:53:42.069579 (+ 2546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:53:42.070254 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:53:42.070259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:53:42.070262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:53:42.070390 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:53:42.070889 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:53:42.070895 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:53:42.070898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:53:42.071182 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:53:42.071320 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 00:53:42.071996 (+ 676us) server_negotiation.cc:299] Negotiation successful
0810 00:53:42.072213 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":209,"threads_started":1}
I20260810 00:53:42.073469 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:42.055673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42413 (local address 127.10.97.66:53705)
0810 00:53:42.056361 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0810 00:53:42.056403 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 00:53:42.057983 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:53:42.058684 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:53:42.058692 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:53:42.059118 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:53:42.059670 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:53:42.059688 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.060794 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.060800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:53:42.062586 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:53:42.062600 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.063554 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.063560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:53:42.063857 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:42.064514 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:53:42.064553 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:53:42.066799 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:53:42.069710 (+ 2911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:53:42.069719 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:53:42.069733 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:53:42.070143 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:53:42.070492 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:53:42.070497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:53:42.070502 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:53:42.070710 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:53:42.071305 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:53:42.071312 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:53:42.071625 (+ 313us) client_negotiation.cc:769] Sending connection context
0810 00:53:42.071961 (+ 336us) client_negotiation.cc:240] Negotiation successful
0810 00:53:42.072375 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":291,"threads_started":1}
I20260810 00:53:42.074685 28571 heartbeater.cc:347] Connected to a master server at 127.10.97.126:42413
I20260810 00:53:42.075001 28571 heartbeater.cc:464] Registering TS with master...
I20260810 00:53:42.075868 28571 heartbeater.cc:511] Master 127.10.97.126:42413 requested a full tablet report, sending...
I20260810 00:53:42.077229 28006 ts_manager.cc:194] Registered new tserver with Master: 38476c2a69db47a7b82d08d3d018aa7d (127.10.97.66:36081)
I20260810 00:53:42.077988 28006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:53705
WARNING: no policy specified for kudu/127.10.97.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
I20260810 00:53:42.114775 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.97.84:56889/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:42413
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:42481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:53:42.435303 28579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:53:42.435560 28579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:53:42.435698 28579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:53:42.442629 28579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:53:42.442735 28579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:53:42.442889 28579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:53:42.452014 28579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:42481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.97.84:56889/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:42413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:53:42.453135 28579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:53:42.454303 28579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 00:53:42.466451 28352 heartbeater.cc:503] Master 127.10.97.126:42413 was elected leader, sending a full tablet report...
W20260810 00:53:42.466509 28591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:53:42.466480 28593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:53:42.467052 28590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:53:42.467717 28579 server_base.cc:1034] running on GCE node
I20260810 00:53:42.468418 28579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:53:42.469286 28579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:53:42.470470 28579 hybrid_clock.cc:648] HybridClock initialized: now 1786323222470415 us; error 47 us; skew 500 ppm
I20260810 00:53:42.470799 28579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:53:42 dist-test-slave-jlzn krb5kdc[27293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323222, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:53:42.473896 28579 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:53:42.475215 28579 webserver.cc:460] Webserver started at http://127.10.97.67:36167/ using document root <none> and password file <none>
I20260810 00:53:42.475890 28579 fs_manager.cc:359] Metadata directory not provided
I20260810 00:53:42.476013 28579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:53:42.476295 28579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 00:53:42 dist-test-slave-jlzn krb5kdc[27293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323222, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
I20260810 00:53:42.493008 28579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "81b6a1dbdc5147eabeaf6e7b3274e0aa"
format_stamp: "Formatted at 2026-08-10 00:53:42 on dist-test-slave-jlzn"
server_key: "1515b18b5c73dc57c12e35b47fdd9e11"
server_key_iv: "b09698be3278e801235e2800d9011801"
server_key_version: "kuduclusterkey@0"
I20260810 00:53:42.494140 28579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "81b6a1dbdc5147eabeaf6e7b3274e0aa"
format_stamp: "Formatted at 2026-08-10 00:53:42 on dist-test-slave-jlzn"
server_key: "1515b18b5c73dc57c12e35b47fdd9e11"
server_key_iv: "b09698be3278e801235e2800d9011801"
server_key_version: "kuduclusterkey@0"
I20260810 00:53:42.500651 28579 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 00:53:42.513491 28607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:53:42.514955 28579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 00:53:42.515151 28579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "81b6a1dbdc5147eabeaf6e7b3274e0aa"
format_stamp: "Formatted at 2026-08-10 00:53:42 on dist-test-slave-jlzn"
server_key: "1515b18b5c73dc57c12e35b47fdd9e11"
server_key_iv: "b09698be3278e801235e2800d9011801"
server_key_version: "kuduclusterkey@0"
I20260810 00:53:42.515376 28579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:53:42.530194 28579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:53:42.533579 28579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:53:42.533841 28579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:53:42.534752 28579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:53:42.536106 28579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:53:42.536192 28579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:53:42.536315 28579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:53:42.536370 28579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:53:42.566334 28579 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:38543
I20260810 00:53:42.566373 28776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:38543 every 8 connection(s)
I20260810 00:53:42.567473 28579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:53:42.574922 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 28579
I20260810 00:53:42.575078 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:53:42.575474 10629 external_mini_cluster.cc:1428] Setting key 3f3f9ba17659f67deb041f9e55f7b43b
Aug 10 00:53:42 dist-test-slave-jlzn krb5kdc[27293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323222, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:53:42.585659 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:42.569773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:47567 (local address 127.10.97.126:42413)
0810 00:53:42.569969 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 00:53:42.569975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:53:42.571229 (+ 1254us) server_negotiation.cc:407] Connection header received
0810 00:53:42.572964 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:53:42.572968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:53:42.573049 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:53:42.573132 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:53:42.574446 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.575203 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.576930 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.577356 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.577463 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:42.580504 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:53:42.580509 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:53:42.580512 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:53:42.580543 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:53:42.582837 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:53:42.583491 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:53:42.583494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:53:42.583497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:53:42.583615 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:53:42.584072 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:53:42.584075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:53:42.584077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:53:42.584343 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:53:42.584479 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 00:53:42.585098 (+ 619us) server_negotiation.cc:299] Negotiation successful
0810 00:53:42.585317 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 00:53:42.586423 28780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:42.570288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42413 (local address 127.10.97.67:47567)
0810 00:53:42.571070 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0810 00:53:42.571120 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 00:53:42.572742 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:53:42.573324 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:53:42.573331 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:53:42.573726 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:53:42.574296 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:53:42.574314 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.575342 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.575345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:53:42.576803 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:53:42.576816 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.577571 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.577576 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:53:42.577779 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:42.578398 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:53:42.578420 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:53:42.580314 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:53:42.582962 (+ 2648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:53:42.582970 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:53:42.582983 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:53:42.583373 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:53:42.583717 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:53:42.583722 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:53:42.583726 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:53:42.583961 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:53:42.584472 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:53:42.584480 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:53:42.584770 (+ 290us) client_negotiation.cc:769] Sending connection context
0810 00:53:42.584980 (+ 210us) client_negotiation.cc:240] Negotiation successful
0810 00:53:42.585416 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":270,"threads_started":1}
I20260810 00:53:42.587527 28777 heartbeater.cc:347] Connected to a master server at 127.10.97.126:42413
I20260810 00:53:42.587787 28777 heartbeater.cc:464] Registering TS with master...
I20260810 00:53:42.588469 28777 heartbeater.cc:511] Master 127.10.97.126:42413 requested a full tablet report, sending...
I20260810 00:53:42.590008 28006 ts_manager.cc:194] Registered new tserver with Master: 81b6a1dbdc5147eabeaf6e7b3274e0aa (127.10.97.67:38543)
I20260810 00:53:42.590597 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:53:42.590935 28006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:47567
I20260810 00:53:42.606483 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:42.595559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49350 (local address 127.10.97.126:42413)
0810 00:53:42.595746 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 00:53:42.595752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:53:42.596128 (+ 376us) server_negotiation.cc:407] Connection header received
0810 00:53:42.596282 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:53:42.596286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:53:42.596339 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:53:42.596405 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:53:42.597717 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.598442 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.599686 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.600134 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.600222 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:42.602233 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:53:42.602238 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:53:42.602241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:53:42.602268 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:53:42.604494 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:53:42.605055 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:53:42.605058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:53:42.605060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:53:42.605190 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:53:42.605466 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:53:42.605469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:53:42.605471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:53:42.605690 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:53:42.605884 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 00:53:42.606067 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 00:53:42.606200 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 00:53:42.610093 28006 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:53:42.616997 28006 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:53:42.653601 28804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:42.628977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35740 (local address 127.10.97.67:38543)
0810 00:53:42.644204 (+ 15227us) server_negotiation.cc:206] Beginning negotiation
0810 00:53:42.644213 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:53:42.645976 (+ 1763us) server_negotiation.cc:407] Connection header received
0810 00:53:42.646077 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:53:42.646084 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:53:42.646312 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:53:42.646509 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:53:42.647240 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.648865 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.651152 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.652510 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.652634 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:42.652858 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 00:53:42.653106 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 00:53:42.653270 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15057,"thread_start_us":15238,"threads_started":1}
I20260810 00:53:42.653602 28802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:42.628439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:38543 (local address 127.0.0.1:35740)
0810 00:53:42.645203 (+ 16764us) negotiation.cc:107] Waiting for socket to connect
0810 00:53:42.645221 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:53:42.645351 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:53:42.646531 (+ 1180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:53:42.646538 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:53:42.646568 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:53:42.646997 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:53:42.647011 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.648997 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.649002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:53:42.650988 (+ 1986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:53:42.651000 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.652704 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.652710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:53:42.652903 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:42.652941 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 00:53:42.653046 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 00:53:42.653220 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16633,"mutex_wait_us":72,"thread_start_us":17212,"threads_started":1}
I20260810 00:53:42.654629 28797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:42.628186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:36081 (local address 127.0.0.1:37094)
0810 00:53:42.643286 (+ 15100us) negotiation.cc:107] Waiting for socket to connect
0810 00:53:42.643336 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 00:53:42.643515 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:53:42.646978 (+ 3463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:53:42.646986 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:53:42.647006 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:53:42.647474 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:53:42.647485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.649170 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.649175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:53:42.651642 (+ 2467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:53:42.651658 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.653731 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.653738 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:53:42.653958 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:42.654043 (+ 85us) client_negotiation.cc:769] Sending connection context
0810 00:53:42.654123 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 00:53:42.654271 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14837,"mutex_wait_us":90,"thread_start_us":15492,"threads_started":1}
I20260810 00:53:42.654644 28799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:42.628373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37094 (local address 127.10.97.66:36081)
0810 00:53:42.646431 (+ 18058us) server_negotiation.cc:206] Beginning negotiation
0810 00:53:42.646439 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:53:42.646466 (+ 27us) server_negotiation.cc:407] Connection header received
0810 00:53:42.646562 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:53:42.646569 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:53:42.646784 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:53:42.646996 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:53:42.647633 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.649019 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.651850 (+ 2831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.653239 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.653369 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:42.653540 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 00:53:42.654203 (+ 663us) server_negotiation.cc:299] Negotiation successful
0810 00:53:42.654359 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17915,"thread_start_us":17818,"threads_started":1}
I20260810 00:53:42.654737 28805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:42.629615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39608 (local address 127.10.97.65:38641)
0810 00:53:42.644777 (+ 15162us) server_negotiation.cc:206] Beginning negotiation
0810 00:53:42.644786 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:53:42.645970 (+ 1184us) server_negotiation.cc:407] Connection header received
0810 00:53:42.646094 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:53:42.646099 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:53:42.646312 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:53:42.646513 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:53:42.647495 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.649032 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.651846 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.653240 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.653369 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:42.653575 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 00:53:42.654288 (+ 713us) server_negotiation.cc:299] Negotiation successful
0810 00:53:42.654437 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14984,"thread_start_us":14800,"threads_started":1}
I20260810 00:53:42.655231 28800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:42.628973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:38641 (local address 127.0.0.1:39608)
0810 00:53:42.645837 (+ 16864us) negotiation.cc:107] Waiting for socket to connect
0810 00:53:42.645858 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:53:42.645973 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:53:42.646786 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:53:42.646789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:53:42.646810 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:53:42.647300 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:53:42.647312 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.649179 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.649183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:53:42.651679 (+ 2496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:53:42.651689 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:53:42.653746 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:53:42.653752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:53:42.653956 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:42.654037 (+ 81us) client_negotiation.cc:769] Sending connection context
0810 00:53:42.654123 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 00:53:42.654915 (+ 792us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16743,"mutex_wait_us":758,"thread_start_us":15962,"threads_started":1}
I20260810 00:53:42.657892 28469 tablet_service.cc:1467] Processing CreateTablet for tablet bc480a74a13b4d639fe7c915c64f2ad4 (DEFAULT_TABLE table=test-table [id=5956b5b7823c4ce8817752214b88c7d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:53:42.658048 28676 tablet_service.cc:1467] Processing CreateTablet for tablet bc480a74a13b4d639fe7c915c64f2ad4 (DEFAULT_TABLE table=test-table [id=5956b5b7823c4ce8817752214b88c7d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:53:42.658478 28259 tablet_service.cc:1467] Processing CreateTablet for tablet bc480a74a13b4d639fe7c915c64f2ad4 (DEFAULT_TABLE table=test-table [id=5956b5b7823c4ce8817752214b88c7d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:53:42.658988 28469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc480a74a13b4d639fe7c915c64f2ad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:53:42.659461 28676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc480a74a13b4d639fe7c915c64f2ad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:53:42.659797 28259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc480a74a13b4d639fe7c915c64f2ad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:53:42.670096 28806 tablet_bootstrap.cc:492] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d: Bootstrap starting.
I20260810 00:53:42.673449 28806 tablet_bootstrap.cc:654] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d: Neither blocks nor log segments found. Creating new log.
I20260810 00:53:42.674559 28806 log.cc:824] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d: Log is configured to *not* fsync() on all Append() calls
I20260810 00:53:42.677114 28808 tablet_bootstrap.cc:492] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa: Bootstrap starting.
I20260810 00:53:42.677796 28806 tablet_bootstrap.cc:492] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d: No bootstrap required, opened a new log
I20260810 00:53:42.678025 28806 ts_tablet_manager.cc:1397] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260810 00:53:42.679919 28808 tablet_bootstrap.cc:654] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa: Neither blocks nor log segments found. Creating new log.
I20260810 00:53:42.680836 28808 log.cc:824] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa: Log is configured to *not* fsync() on all Append() calls
I20260810 00:53:42.676247 28807 tablet_bootstrap.cc:492] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9: Bootstrap starting.
I20260810 00:53:42.683308 28808 tablet_bootstrap.cc:492] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa: No bootstrap required, opened a new log
I20260810 00:53:42.683564 28808 ts_tablet_manager.cc:1397] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260810 00:53:42.685950 28806 raft_consensus.cc:348] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38476c2a69db47a7b82d08d3d018aa7d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36081 } } peers { permanent_uuid: "81b6a1dbdc5147eabeaf6e7b3274e0aa" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38543 } } peers { permanent_uuid: "98c2d9e1b3d2435d9348aca755a877f9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38641 } }
I20260810 00:53:42.686383 28806 raft_consensus.cc:374] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:53:42.686482 28807 tablet_bootstrap.cc:654] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9: Neither blocks nor log segments found. Creating new log.
I20260810 00:53:42.686491 28806 raft_consensus.cc:729] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38476c2a69db47a7b82d08d3d018aa7d, State: Initialized, Role: FOLLOWER
I20260810 00:53:42.687099 28806 consensus_queue.cc:260] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38476c2a69db47a7b82d08d3d018aa7d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36081 } } peers { permanent_uuid: "81b6a1dbdc5147eabeaf6e7b3274e0aa" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38543 } } peers { permanent_uuid: "98c2d9e1b3d2435d9348aca755a877f9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38641 } }
I20260810 00:53:42.687695 28807 log.cc:824] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9: Log is configured to *not* fsync() on all Append() calls
I20260810 00:53:42.688279 28571 heartbeater.cc:503] Master 127.10.97.126:42413 was elected leader, sending a full tablet report...
I20260810 00:53:42.688819 28806 ts_tablet_manager.cc:1428] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260810 00:53:42.691279 28808 raft_consensus.cc:348] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38476c2a69db47a7b82d08d3d018aa7d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36081 } } peers { permanent_uuid: "81b6a1dbdc5147eabeaf6e7b3274e0aa" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38543 } } peers { permanent_uuid: "98c2d9e1b3d2435d9348aca755a877f9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38641 } }
I20260810 00:53:42.691697 28808 raft_consensus.cc:374] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:53:42.691829 28808 raft_consensus.cc:729] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81b6a1dbdc5147eabeaf6e7b3274e0aa, State: Initialized, Role: FOLLOWER
I20260810 00:53:42.692103 28807 tablet_bootstrap.cc:492] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9: No bootstrap required, opened a new log
I20260810 00:53:42.692376 28807 ts_tablet_manager.cc:1397] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.000s
I20260810 00:53:42.692414 28808 consensus_queue.cc:260] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38476c2a69db47a7b82d08d3d018aa7d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36081 } } peers { permanent_uuid: "81b6a1dbdc5147eabeaf6e7b3274e0aa" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38543 } } peers { permanent_uuid: "98c2d9e1b3d2435d9348aca755a877f9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38641 } }
I20260810 00:53:42.694651 28777 heartbeater.cc:503] Master 127.10.97.126:42413 was elected leader, sending a full tablet report...
I20260810 00:53:42.695281 28808 ts_tablet_manager.cc:1428] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260810 00:53:42.700421 28807 raft_consensus.cc:348] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38476c2a69db47a7b82d08d3d018aa7d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36081 } } peers { permanent_uuid: "81b6a1dbdc5147eabeaf6e7b3274e0aa" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38543 } } peers { permanent_uuid: "98c2d9e1b3d2435d9348aca755a877f9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38641 } }
I20260810 00:53:42.700709 28807 raft_consensus.cc:374] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:53:42.700843 28807 raft_consensus.cc:729] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98c2d9e1b3d2435d9348aca755a877f9, State: Initialized, Role: FOLLOWER
I20260810 00:53:42.701290 28807 consensus_queue.cc:260] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38476c2a69db47a7b82d08d3d018aa7d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36081 } } peers { permanent_uuid: "81b6a1dbdc5147eabeaf6e7b3274e0aa" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38543 } } peers { permanent_uuid: "98c2d9e1b3d2435d9348aca755a877f9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38641 } }
I20260810 00:53:42.702428 28807 ts_tablet_manager.cc:1428] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
W20260810 00:53:42.804729 28572 tablet.cc:2369] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:53:42.818936 28779 tablet.cc:2369] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:53:42.899881 28353 tablet.cc:2369] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:53:43.049418 28819 raft_consensus.cc:482] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:53:43.049713 28819 raft_consensus.cc:504] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38476c2a69db47a7b82d08d3d018aa7d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36081 } } peers { permanent_uuid: "81b6a1dbdc5147eabeaf6e7b3274e0aa" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38543 } } peers { permanent_uuid: "98c2d9e1b3d2435d9348aca755a877f9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38641 } }
I20260810 00:53:43.051395 28819 leader_election.cc:288] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38476c2a69db47a7b82d08d3d018aa7d (127.10.97.66:36081), 81b6a1dbdc5147eabeaf6e7b3274e0aa (127.10.97.67:38543)
I20260810 00:53:43.059371 28804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:43.052037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:41861 (local address 127.10.97.67:38543)
0810 00:53:43.053343 (+ 1306us) server_negotiation.cc:206] Beginning negotiation
0810 00:53:43.053349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:53:43.053371 (+ 22us) server_negotiation.cc:407] Connection header received
0810 00:53:43.053574 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:53:43.053579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:53:43.053652 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:53:43.053740 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:53:43.054348 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:43.055387 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:43.057693 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:43.058443 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:43.058588 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:43.058671 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 00:53:43.058986 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 00:53:43.059119 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1176}
I20260810 00:53:43.059417 28821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:43.051886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:38543 (local address 127.10.97.65:41861)
0810 00:53:43.053311 (+ 1425us) negotiation.cc:107] Waiting for socket to connect
0810 00:53:43.053327 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:53:43.053430 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:53:43.053771 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:53:43.053776 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:53:43.053800 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:53:43.054179 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:53:43.054191 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:53:43.055736 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:53:43.055742 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:53:43.057539 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:53:43.057550 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:53:43.058634 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:53:43.058640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:53:43.058812 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:43.058844 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 00:53:43.058944 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 00:53:43.059066 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1304,"thread_start_us":421,"threads_started":1}
I20260810 00:53:43.059553 28799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:43.052215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:40455 (local address 127.10.97.66:36081)
0810 00:53:43.052433 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 00:53:43.052439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:53:43.052727 (+ 288us) server_negotiation.cc:407] Connection header received
0810 00:53:43.052947 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:53:43.052953 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:53:43.053039 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:53:43.053129 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:53:43.053872 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:43.054748 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:43.057148 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:43.058495 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:43.058639 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:43.058710 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 00:53:43.059178 (+ 468us) server_negotiation.cc:299] Negotiation successful
0810 00:53:43.059313 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 00:53:43.060309 28820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:43.051660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:36081 (local address 127.10.97.65:40455)
0810 00:53:43.052619 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0810 00:53:43.052635 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:53:43.052776 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:53:43.053305 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:53:43.053310 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:53:43.053338 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:53:43.053711 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:53:43.053724 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:53:43.054884 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:53:43.054887 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:53:43.056983 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:53:43.056998 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:53:43.058837 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:53:43.058843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:53:43.059005 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:43.059036 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 00:53:43.059718 (+ 682us) client_negotiation.cc:240] Negotiation successful
0810 00:53:43.059868 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"mutex_wait_us":1,"spinlock_wait_cycles":17280,"thread_start_us":1346,"threads_started":1}
I20260810 00:53:43.060729 28703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc480a74a13b4d639fe7c915c64f2ad4" candidate_uuid: "98c2d9e1b3d2435d9348aca755a877f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81b6a1dbdc5147eabeaf6e7b3274e0aa" is_pre_election: true
I20260810 00:53:43.061234 28703 raft_consensus.cc:2454] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98c2d9e1b3d2435d9348aca755a877f9 in term 0.
I20260810 00:53:43.061297 28498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc480a74a13b4d639fe7c915c64f2ad4" candidate_uuid: "98c2d9e1b3d2435d9348aca755a877f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38476c2a69db47a7b82d08d3d018aa7d" is_pre_election: true
I20260810 00:53:43.061842 28498 raft_consensus.cc:2454] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98c2d9e1b3d2435d9348aca755a877f9 in term 0.
I20260810 00:53:43.061911 28188 leader_election.cc:302] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81b6a1dbdc5147eabeaf6e7b3274e0aa, 98c2d9e1b3d2435d9348aca755a877f9; no voters:
I20260810 00:53:43.062337 28819 raft_consensus.cc:2781] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:53:43.062490 28819 raft_consensus.cc:482] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:53:43.062608 28819 raft_consensus.cc:3037] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:53:43.065758 28819 raft_consensus.cc:504] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38476c2a69db47a7b82d08d3d018aa7d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36081 } } peers { permanent_uuid: "81b6a1dbdc5147eabeaf6e7b3274e0aa" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38543 } } peers { permanent_uuid: "98c2d9e1b3d2435d9348aca755a877f9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38641 } }
I20260810 00:53:43.066305 28819 leader_election.cc:288] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [CANDIDATE]: Term 1 election: Requested vote from peers 38476c2a69db47a7b82d08d3d018aa7d (127.10.97.66:36081), 81b6a1dbdc5147eabeaf6e7b3274e0aa (127.10.97.67:38543)
I20260810 00:53:43.066737 28498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc480a74a13b4d639fe7c915c64f2ad4" candidate_uuid: "98c2d9e1b3d2435d9348aca755a877f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38476c2a69db47a7b82d08d3d018aa7d"
I20260810 00:53:43.066744 28703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc480a74a13b4d639fe7c915c64f2ad4" candidate_uuid: "98c2d9e1b3d2435d9348aca755a877f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81b6a1dbdc5147eabeaf6e7b3274e0aa"
I20260810 00:53:43.066905 28498 raft_consensus.cc:3037] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:53:43.066917 28703 raft_consensus.cc:3037] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:53:43.068974 28498 raft_consensus.cc:2454] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98c2d9e1b3d2435d9348aca755a877f9 in term 1.
I20260810 00:53:43.069025 28703 raft_consensus.cc:2454] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98c2d9e1b3d2435d9348aca755a877f9 in term 1.
I20260810 00:53:43.069346 28189 leader_election.cc:302] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38476c2a69db47a7b82d08d3d018aa7d, 98c2d9e1b3d2435d9348aca755a877f9; no voters:
I20260810 00:53:43.069621 28819 raft_consensus.cc:2781] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:53:43.070094 28819 raft_consensus.cc:686] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [term 1 LEADER]: Becoming Leader. State: Replica: 98c2d9e1b3d2435d9348aca755a877f9, State: Running, Role: LEADER
I20260810 00:53:43.070596 28819 consensus_queue.cc:237] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38476c2a69db47a7b82d08d3d018aa7d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36081 } } peers { permanent_uuid: "81b6a1dbdc5147eabeaf6e7b3274e0aa" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38543 } } peers { permanent_uuid: "98c2d9e1b3d2435d9348aca755a877f9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38641 } }
I20260810 00:53:43.074885 28005 catalog_manager.cc:5495] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 98c2d9e1b3d2435d9348aca755a877f9 (127.10.97.65). New cstate: current_term: 1 leader_uuid: "98c2d9e1b3d2435d9348aca755a877f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38476c2a69db47a7b82d08d3d018aa7d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81b6a1dbdc5147eabeaf6e7b3274e0aa" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98c2d9e1b3d2435d9348aca755a877f9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38641 } health_report { overall_health: HEALTHY } } }
I20260810 00:53:43.104271 28805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:43.099036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39616 (local address 127.10.97.65:38641)
0810 00:53:43.099224 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 00:53:43.099230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:53:43.099299 (+ 69us) server_negotiation.cc:407] Connection header received
0810 00:53:43.099527 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:53:43.099531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:53:43.099603 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:53:43.099729 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:53:43.100122 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:43.100865 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:43.102286 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:43.102673 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:43.102779 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:43.103267 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:53:43.103714 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:53:43.103872 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 00:53:43.103954 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 00:53:43.104032 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 00:53:43.111527 28498 raft_consensus.cc:1261] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d [term 1 FOLLOWER]: Refusing update from remote peer 98c2d9e1b3d2435d9348aca755a877f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:53:43.111531 28703 raft_consensus.cc:1261] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa [term 1 FOLLOWER]: Refusing update from remote peer 98c2d9e1b3d2435d9348aca755a877f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:53:43.112383 28822 consensus_queue.cc:1035] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81b6a1dbdc5147eabeaf6e7b3274e0aa" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:53:43.112737 28819 consensus_queue.cc:1035] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38476c2a69db47a7b82d08d3d018aa7d" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:53:43.122337 28827 mvcc.cc:205] Tried to move back new op lower bound from 7316779921856110592 to 7316779921700761600. Current Snapshot: MvccSnapshot[applied={T|T < 7316779921856110592}]
I20260810 00:53:43.123487 28829 mvcc.cc:205] Tried to move back new op lower bound from 7316779921856110592 to 7316779921700761600. Current Snapshot: MvccSnapshot[applied={T|T < 7316779921856110592}]
I20260810 00:53:43.134480 28005 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:53:43.134729 28005 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:53:43.138070 28005 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c12204e4cf59472e8577b0208224c8c9: Sending DeleteTablet for 3 replicas of tablet bc480a74a13b4d639fe7c915c64f2ad4
I20260810 00:53:43.139117 28469 tablet_service.cc:1514] Processing DeleteTablet for tablet bc480a74a13b4d639fe7c915c64f2ad4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:53:43 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:37094
I20260810 00:53:43.139132 28259 tablet_service.cc:1514] Processing DeleteTablet for tablet bc480a74a13b4d639fe7c915c64f2ad4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:53:43 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:39608
I20260810 00:53:43.139384 28676 tablet_service.cc:1514] Processing DeleteTablet for tablet bc480a74a13b4d639fe7c915c64f2ad4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:53:43 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:35740
I20260810 00:53:43.140069 28842 tablet_replica.cc:330] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d: stopping tablet replica
I20260810 00:53:43.140525 28842 raft_consensus.cc:2229] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:53:43.141032 28842 raft_consensus.cc:2258] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:53:43.141621 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 28151
I20260810 00:53:43.141831 28841 tablet_replica.cc:330] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9: stopping tablet replica
I20260810 00:53:43.142206 28841 raft_consensus.cc:2229] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:53:43.142828 28841 raft_consensus.cc:2258] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:53:43.144342 28842 ts_tablet_manager.cc:1905] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:53:43.144716 28841 ts_tablet_manager.cc:1905] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:53:43.152254 28841 ts_tablet_manager.cc:1918] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:53:43.152483 28841 log.cc:1196] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/bc480a74a13b4d639fe7c915c64f2ad4
I20260810 00:53:43.153079 28841 ts_tablet_manager.cc:1939] T bc480a74a13b4d639fe7c915c64f2ad4 P 98c2d9e1b3d2435d9348aca755a877f9: Deleting consensus metadata
I20260810 00:53:43.154938 27982 catalog_manager.cc:4841] TS 98c2d9e1b3d2435d9348aca755a877f9 (127.10.97.65:38641): tablet bc480a74a13b4d639fe7c915c64f2ad4 (table test-table [id=5956b5b7823c4ce8817752214b88c7d7]) successfully deleted
I20260810 00:53:43.156683 28843 tablet_replica.cc:330] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa: stopping tablet replica
I20260810 00:53:43.157035 28843 raft_consensus.cc:2229] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:53:43.157434 28843 raft_consensus.cc:2258] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:53:43.157757 28842 ts_tablet_manager.cc:1918] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:53:43.157950 28842 log.cc:1196] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/bc480a74a13b4d639fe7c915c64f2ad4
I20260810 00:53:43.158438 28842 ts_tablet_manager.cc:1939] T bc480a74a13b4d639fe7c915c64f2ad4 P 38476c2a69db47a7b82d08d3d018aa7d: Deleting consensus metadata
I20260810 00:53:43.159917 28843 ts_tablet_manager.cc:1905] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:53:43.160192 27985 catalog_manager.cc:4841] TS 38476c2a69db47a7b82d08d3d018aa7d (127.10.97.66:36081): tablet bc480a74a13b4d639fe7c915c64f2ad4 (table test-table [id=5956b5b7823c4ce8817752214b88c7d7]) successfully deleted
I20260810 00:53:43.166062 28843 ts_tablet_manager.cc:1918] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:53:43.166261 28843 log.cc:1196] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/bc480a74a13b4d639fe7c915c64f2ad4
I20260810 00:53:43.166785 28843 ts_tablet_manager.cc:1939] T bc480a74a13b4d639fe7c915c64f2ad4 P 81b6a1dbdc5147eabeaf6e7b3274e0aa: Deleting consensus metadata
Aug 10 00:53:43 dist-test-slave-jlzn krb5kdc[27293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:53:43.168282 27984 catalog_manager.cc:4841] TS 81b6a1dbdc5147eabeaf6e7b3274e0aa (127.10.97.67:38543): tablet bc480a74a13b4d639fe7c915c64f2ad4 (table test-table [id=5956b5b7823c4ce8817752214b88c7d7]) successfully deleted
I20260810 00:53:43.176017 28805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:43.142543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39630 (local address 127.10.97.65:38641)
0810 00:53:43.142729 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 00:53:43.142735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:53:43.157531 (+ 14796us) server_negotiation.cc:407] Connection header received
0810 00:53:43.157779 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:53:43.157784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:53:43.157863 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:53:43.157961 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:53:43.159757 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:43.161045 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:43.162917 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:43.163610 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:43.163744 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:43.169278 (+ 5534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:53:43.169292 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:53:43.169309 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:53:43.169355 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:53:43.173282 (+ 3927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:53:43.173889 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:53:43.173898 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:53:43.173902 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:53:43.174079 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:53:43.174399 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:53:43.174403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:53:43.174407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:53:43.174878 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:53:43.175187 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0810 00:53:43.175337 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 00:53:43.175581 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 00:53:43.176800 28341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:53:43.275394 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 28151
I20260810 00:53:43.302776 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 28358
Aug 10 00:53:43 dist-test-slave-jlzn krb5kdc[27293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.66@KRBTEST.COM
I20260810 00:53:43.317229 28799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:43.303536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37102 (local address 127.10.97.66:36081)
0810 00:53:43.303704 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 00:53:43.303708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:53:43.303743 (+ 35us) server_negotiation.cc:407] Connection header received
0810 00:53:43.303932 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:53:43.303936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:53:43.304004 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:53:43.304088 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:53:43.305265 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:43.306047 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:43.307235 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:43.307635 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:43.307728 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:43.311907 (+ 4179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:53:43.311921 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:53:43.311936 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:53:43.311967 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:53:43.314641 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:53:43.315306 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:53:43.315313 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:53:43.315315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:53:43.315444 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:53:43.315850 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:53:43.315854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:53:43.315857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:53:43.316267 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:53:43.316498 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0810 00:53:43.316761 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 00:53:43.316958 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 00:53:43.317855 28565 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:53:43.404044 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 28358
I20260810 00:53:43.422462 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 28579
Aug 10 00:53:43 dist-test-slave-jlzn krb5kdc[27293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.67@KRBTEST.COM
I20260810 00:53:43.436988 28804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:53:43.423148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35748 (local address 127.10.97.67:38543)
0810 00:53:43.423301 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 00:53:43.423305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:53:43.423371 (+ 66us) server_negotiation.cc:407] Connection header received
0810 00:53:43.423518 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:53:43.423522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:53:43.423588 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:53:43.423675 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:53:43.425028 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:43.425750 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:43.426935 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:53:43.427305 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:53:43.427392 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:53:43.431450 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:53:43.431471 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:53:43.431506 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:53:43.431552 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:53:43.434274 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:53:43.434920 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:53:43.434929 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:53:43.434931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:53:43.435083 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:53:43.435429 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:53:43.435433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:53:43.435435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:53:43.435883 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:53:43.436159 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 00:53:43.436465 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 00:53:43.436681 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 00:53:43.437582 28766 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:53:43.523087 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 28579
I20260810 00:53:43.545821 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 27962
I20260810 00:53:43.546969 28058 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:53:43.654364 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 27962
I20260810 00:53:43.681121 10629 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 00:53:44.025753 10629 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 00:53:44.026039 10629 mini_ranger.cc:66] Stopping Ranger...
I20260810 00:53:44.369549 10629 mini_ranger.cc:68] Stopped Ranger
2026-08-10 00:53:44.369 UTC [27339] LOG: received smart shutdown request
2026-08-10 00:53:44.373 UTC [27339] LOG: background worker "logical replication launcher" (PID 27354) exited with exit code 1
2026-08-10 00:53:44.373 UTC [27347] LOG: shutting down
2026-08-10 00:53:44.494 UTC [27339] LOG: database system is shut down
2026-08-10T00:53:44Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47874 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:53:44 dist-test-slave-jlzn krb5kdc[28864](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:53:44 dist-test-slave-jlzn krb5kdc[28864](info): set up 2 sockets
Aug 10 00:53:44 dist-test-slave-jlzn krb5kdc[28864](info): commencing operation
krb5kdc: starting...
W20260810 00:53:45.801997 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:53:45 dist-test-slave-jlzn krb5kdc[28864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:53:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:53:45Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.97.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.97.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.97.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/rangeradmin_127.10.97.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.97.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/rangeradmin_127.10.97.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.97.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.97.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.97.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/rangerlookup_127.10.97.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.97.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/rangerlookup_127.10.97.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.97.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.97.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.97.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/HTTP_127.10.97.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.97.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/HTTP_127.10.97.84@KRBTEST.COM.keytab.
I20260810 00:53:46.112641 10629 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskH92XUb/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/postgres -l logfile start
2026-08-10 00:53:48.254 UTC [28908] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 00:53:48.254 UTC [28908] LOG: listening on IPv4 address "127.10.97.84", port 47437
2026-08-10 00:53:48.257 UTC [28908] LOG: listening on Unix socket "/tmp/.s.PGSQL.47437"
2026-08-10 00:53:48.277 UTC [28917] LOG: database system was shut down at 2026-08-10 00:53:47 UTC
2026-08-10 00:53:48.280 UTC [28908] LOG: database system is ready to accept connections
I20260810 00:53:49.345633 10629 mini_postgres.cc:96] Postgres bound to 47437
2026-08-10 00:53:49.366 UTC [28934] FATAL: database "slave" does not exist
127.10.97.84:47437 - accepting connections
I20260810 00:53:49.367285 10629 mini_ranger.cc:161] Starting Ranger...
I20260810 00:53:49.392691 10629 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 00:53:49.769676 10629 mini_ranger.cc:87] Created ranger Postgres database
I20260810 00:53:49.769778 10629 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ranger-admin
2026-08-10 00:53:50,048 [I] DB FLAVOR :POSTGRES
2026-08-10 00:53:50,049 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 00:53:50,049 [I] Checking connection..
2026-08-10 00:53:50,049 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 00:53:50,306 [I] Checking connection passed.
2026-08-10 00:53:50,307 [I] --------- Verifying version history table ---------
2026-08-10 00:53:50,307 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 00:53:50,561 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 00:53:50,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 00:53:50,802 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 00:53:50,802 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 00:53:50,802 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 00:53:51.051 UTC [29029] WARNING: there is no transaction in progress
2026-08-10 00:53:51,059 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 00:53:51,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 00:53:51,286 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 00:53:51,286 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 00:53:51,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 00:53:51,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 00:53:51,772 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-jlzn.c.gcp-upstream.internal','N') ;"
2026-08-10 00:53:52,006 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 00:53:52,259 [I] Table x_portal_user does not exist in database ranger
2026-08-10 00:53:52,259 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 00:53:52,514 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 00:53:52,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 00:53:52,767 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 00:53:53,002 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 00:53:53,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 00:53:53.842 UTC [29183] WARNING: there is no transaction in progress
2026-08-10 00:53:53.856 UTC [29183] WARNING: there is no transaction in progress
2026-08-10 00:53:53.870 UTC [29183] WARNING: there is no transaction in progress
2026-08-10 00:53:53.883 UTC [29183] WARNING: there is no transaction in progress
2026-08-10 00:53:53.896 UTC [29183] WARNING: there is no transaction in progress
2026-08-10 00:53:53.910 UTC [29183] WARNING: there is no transaction in progress
2026-08-10 00:53:53.957 UTC [29183] WARNING: there is no transaction in progress
2026-08-10 00:53:53.963 UTC [29183] WARNING: there is no transaction in progress
2026-08-10 00:53:53.975 UTC [29183] WARNING: there is no transaction in progress
2026-08-10 00:53:53.983 UTC [29183] WARNING: there is no transaction in progress
2026-08-10 00:53:53.992 UTC [29183] WARNING: there is no transaction in progress
2026-08-10 00:53:54.004 UTC [29183] WARNING: there is no transaction in progress
2026-08-10 00:53:54.012 UTC [29183] WARNING: there is no transaction in progress
2026-08-10 00:53:54.017 UTC [29183] WARNING: there is no transaction in progress
2026-08-10 00:53:54.022 UTC [29183] WARNING: there is no transaction in progress
2026-08-10 00:53:54.028 UTC [29183] WARNING: there is no transaction in progress
2026-08-10 00:53:54,545 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 00:53:54,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 00:53:54,773 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 00:53:54,773 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 00:53:55,017 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 00:53:55,018 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 00:53:55,260 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 00:53:55,260 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 00:53:55,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 00:53:55,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-jlzn.c.gcp-upstream.internal';"
2026-08-10 00:53:55,952 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 00:53:55,952 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 00:53:56,180 [I] DB_PATCHES have already been applied
I20260810 00:53:56.186066 10629 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ranger-admin:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskH92XUb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 00:53:56.186161 10629 mini_ranger.cc:193] Using host: 127.10.97.84
I20260810 00:53:56.204537 10629 mini_ranger.cc:227] Ranger admin URL: http://127.10.97.84:56599
Aug 10, 2026 12:53:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 12:53:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 12:53:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 12:53:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 12:53:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 12:53:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 12:53:57 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 00:53:57.443439 10629 mini_ranger.cc:160] Time spent starting Ranger: real 8.076s user 0.005s sys 0.043s
Aug 10, 2026 12:54:15 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 12:54:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Aug 10, 2026 12:54:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 12:54:15 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 12:54:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 12:54:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 12:54:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 12:54:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 12:54:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 12:54:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 12:54:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 12:54:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 12:54:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 12:54:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 12:54:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 12:54:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 12:54:15 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 12:54:16 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 00:54:17.184500 10629 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.97.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.97.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.97.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/rangerkms_127.10.97.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.97.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/rangerkms_127.10.97.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.97.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.97.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.97.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/HTTP_127.10.97.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.97.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/HTTP_127.10.97.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 00:54:17 dist-test-slave-jlzn krb5kdc[28864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323257, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 00:54:17.372941 10629 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 00:54:17.395908 10629 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 00:54:18.110116 10629 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 00:54:18.110258 10629 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ranger-kms
I20260810 00:54:18.110296 10629 mini_ranger_kms.cc:227] Using postgres at 127.10.97.84:47437
2026-08-10 00:54:18,434 [I] DB FLAVOR :POSTGRES
2026-08-10 00:54:18,434 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 00:54:18,434 [I] Checking connection
2026-08-10 00:54:18,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 00:54:18,656 [I] connection success
2026-08-10 00:54:18,656 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 00:54:18,656 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 00:54:18,656 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 00:54:18,887 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 00:54:18,888 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 00:54:18,888 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 00:54:18,888 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.97.84:47437/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 00:54:19,156 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 00:54:19.609781 10629 mini_ranger_kms.cc:310] Created kms service
I20260810 00:54:19.813949 10629 mini_ranger_kms.cc:326] Created kudu user
I20260810 00:54:19.854498 10629 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 00:54:19 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323257, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
I20260810 00:54:20.243916 10629 mini_ranger_kms.cc:384] Added ranger policy
I20260810 00:54:20.244249 10629 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ranger-kms:/tmp/dist-test-taskH92XUb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskH92XUb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 00:54:20.244318 10629 mini_ranger_kms.cc:240] Using host: 127.10.97.84
I20260810 00:54:20.265367 10629 mini_ranger_kms.cc:276] Ranger KMS PID: 29468
I20260810 00:54:20.265482 10629 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.97.84:40887
Aug 10, 2026 12:54:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 12:54:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 12:54:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 12:54:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 12:54:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 12:54:21 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 00:54:22 dist-test-slave-jlzn krb5kdc[28864](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786323262, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.97.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:22.738829 10629 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.366s user 0.057s sys 0.296s
I20260810 00:54:22.739091 10629 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 00:54:22.739293 10629 mini_ranger_kms.cc:401] 127.10.97.84:40887/kms/v1/keys
Aug 10 00:54:22 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323262, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.97.84@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
Aug 10 00:54:23 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323262, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.97.84@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
Aug 10 00:54:25 dist-test-slave-jlzn krb5kdc[28864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:54:25.485589 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:46877
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:42349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:46877
--ranger_config_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.97.84:40887/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:54:25.800254 29545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:25.800515 29545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:25.800608 29545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:25.807395 29545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:54:25.807487 29545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:25.807543 29545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:25.807598 29545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:54:25.807646 29545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:54:25.817026 29545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:42349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.97.84:40887/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:46877
--ranger_config_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:46877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:25.818095 29545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:25.819260 29545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:25.830124 29558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:25.830178 29556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:25.830210 29554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:25.830910 29545 server_base.cc:1034] running on GCE node
I20260810 00:54:25.831629 29545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:25.832887 29545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:25.834084 29545 hybrid_clock.cc:648] HybridClock initialized: now 1786323265834035 us; error 37 us; skew 500 ppm
I20260810 00:54:25.834393 29545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:25 dist-test-slave-jlzn krb5kdc[28864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323265, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:25.837280 29545 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:54:25.838521 29545 webserver.cc:460] Webserver started at http://127.10.97.126:44391/ using document root <none> and password file <none>
I20260810 00:54:25.839074 29545 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:25.839191 29545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:25.839463 29545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 00:54:25 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323265, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
I20260810 00:54:25.895515 29545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "118001e1cd7943468314202ab030828a"
format_stamp: "Formatted at 2026-08-10 00:54:25 on dist-test-slave-jlzn"
server_key: "104c605cb116f56df638f0efde29e343"
server_key_iv: "79290a8ed0176a15184e1e017474c47c"
server_key_version: "kuduclusterkey@0"
I20260810 00:54:25.896689 29545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "118001e1cd7943468314202ab030828a"
format_stamp: "Formatted at 2026-08-10 00:54:25 on dist-test-slave-jlzn"
server_key: "104c605cb116f56df638f0efde29e343"
server_key_iv: "79290a8ed0176a15184e1e017474c47c"
server_key_version: "kuduclusterkey@0"
I20260810 00:54:25.903623 29545 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260810 00:54:25.922967 29569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:25.924360 29545 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 00:54:25.924535 29545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "118001e1cd7943468314202ab030828a"
format_stamp: "Formatted at 2026-08-10 00:54:25 on dist-test-slave-jlzn"
server_key: "104c605cb116f56df638f0efde29e343"
server_key_iv: "79290a8ed0176a15184e1e017474c47c"
server_key_version: "kuduclusterkey@0"
I20260810 00:54:25.924738 29545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:25.937695 29545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:25.940969 29545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:25.941257 29545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:25.957471 29545 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:46877
I20260810 00:54:25.957494 29652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:46877 every 8 connection(s)
I20260810 00:54:25.958644 29545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:54:25.959246 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 29545
I20260810 00:54:25.959447 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:54:25.960227 10629 external_mini_cluster.cc:1428] Setting key 3a664a769b3cdf47dc12dac5f403c969
I20260810 00:54:25.962674 29653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 00:54:25 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:54:25.974673 29653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a: Bootstrap starting.
I20260810 00:54:25.978262 29653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a: Neither blocks nor log segments found. Creating new log.
I20260810 00:54:25.979422 29653 log.cc:824] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a: Log is configured to *not* fsync() on all Append() calls
I20260810 00:54:25.982481 29653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a: No bootstrap required, opened a new log
I20260810 00:54:25.982926 29658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:25.962030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45076 (local address 127.10.97.126:46877)
0810 00:54:25.962799 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:25.962812 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:25.962861 (+ 49us) server_negotiation.cc:407] Connection header received
0810 00:54:25.965025 (+ 2164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:25.965049 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:25.965507 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:25.965916 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:25.967163 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:25.969184 (+ 2021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:25.970496 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:25.971486 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:25.971691 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:25.975420 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:25.975430 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:25.975445 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:25.975484 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:25.978981 (+ 3497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:25.979575 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:25.979582 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:25.979589 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:25.979780 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:25.980239 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:25.980242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:25.980244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:25.980748 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:25.981021 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:25.981374 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 00:54:25.981731 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":318,"threads_started":1}
I20260810 00:54:25.990031 29653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118001e1cd7943468314202ab030828a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 46877 } }
I20260810 00:54:25.990325 29653 raft_consensus.cc:374] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:54:25.990396 29653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 118001e1cd7943468314202ab030828a, State: Initialized, Role: FOLLOWER
I20260810 00:54:25.990862 29653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118001e1cd7943468314202ab030828a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 46877 } }
I20260810 00:54:25.991065 29653 raft_consensus.cc:388] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:54:25.991158 29653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:54:25.991282 29653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:54:25.993175 29653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118001e1cd7943468314202ab030828a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 46877 } }
I20260810 00:54:25.993575 29653 leader_election.cc:302] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 118001e1cd7943468314202ab030828a; no voters:
I20260810 00:54:25.994098 29653 leader_election.cc:288] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:54:25.994247 29662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:54:25.995167 29662 raft_consensus.cc:686] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [term 1 LEADER]: Becoming Leader. State: Replica: 118001e1cd7943468314202ab030828a, State: Running, Role: LEADER
I20260810 00:54:25.995607 29662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118001e1cd7943468314202ab030828a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 46877 } }
I20260810 00:54:25.995981 29653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:54:25.998097 29664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "118001e1cd7943468314202ab030828a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118001e1cd7943468314202ab030828a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 46877 } } }
I20260810 00:54:25.998409 29664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [sys.catalog]: This master's current role is: LEADER
I20260810 00:54:25.998761 29665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 118001e1cd7943468314202ab030828a. Latest consensus state: current_term: 1 leader_uuid: "118001e1cd7943468314202ab030828a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118001e1cd7943468314202ab030828a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 46877 } } }
I20260810 00:54:25.998982 29665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [sys.catalog]: This master's current role is: LEADER
I20260810 00:54:26.003592 29671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:54:26.004082 29653 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 00:54:26.016325 29671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:54:26.033366 29671 catalog_manager.cc:1347] Generated new cluster ID: 580e0988b14d4b56b30868fd6641da88
I20260810 00:54:26.033511 29671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:54:26.052796 29671 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:54:26.054234 29671 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:54:26.059749 29671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a: Generated new TSK 0
I20260810 00:54:26.060484 29671 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
I20260810 00:54:26.757126 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.97.84:40887/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:46877
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:42349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 00:54:27 dist-test-slave-jlzn krb5kdc[28864](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786323267, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 00:54:27.168764 29721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:27.169137 29721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:27.169306 29721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:27.180467 29721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:27.180603 29721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:27.180853 29721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:54:27.195737 29721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:42349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.97.84:40887/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:46877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:27.197607 29721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:27.199417 29721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:27.215718 29739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:27.217164 29742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:27.225368 29721 server_base.cc:1034] running on GCE node
W20260810 00:54:27.225941 29744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:27.226686 29721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:27.227972 29721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:27.229279 29721 hybrid_clock.cc:648] HybridClock initialized: now 1786323267229222 us; error 40 us; skew 500 ppm
I20260810 00:54:27.229656 29721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:27 dist-test-slave-jlzn krb5kdc[28864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323267, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:27.233805 29721 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:54:27.241142 29721 webserver.cc:460] Webserver started at http://127.10.97.65:38415/ using document root <none> and password file <none>
I20260810 00:54:27.242084 29721 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:27.242235 29721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:27.242619 29721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 00:54:27 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323267, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
I20260810 00:54:27.262121 29721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "843b48a518084bd4bfeada56d9f0a671"
format_stamp: "Formatted at 2026-08-10 00:54:27 on dist-test-slave-jlzn"
server_key: "8146f5626c92e97baa147ff2f64dad70"
server_key_iv: "23ab428dcb091a801960eeb2aedae8e1"
server_key_version: "kuduclusterkey@0"
I20260810 00:54:27.263219 29721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "843b48a518084bd4bfeada56d9f0a671"
format_stamp: "Formatted at 2026-08-10 00:54:27 on dist-test-slave-jlzn"
server_key: "8146f5626c92e97baa147ff2f64dad70"
server_key_iv: "23ab428dcb091a801960eeb2aedae8e1"
server_key_version: "kuduclusterkey@0"
I20260810 00:54:27.271711 29721 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260810 00:54:27.292100 29754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:27.293876 29721 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.005s sys 0.001s
I20260810 00:54:27.294107 29721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "843b48a518084bd4bfeada56d9f0a671"
format_stamp: "Formatted at 2026-08-10 00:54:27 on dist-test-slave-jlzn"
server_key: "8146f5626c92e97baa147ff2f64dad70"
server_key_iv: "23ab428dcb091a801960eeb2aedae8e1"
server_key_version: "kuduclusterkey@0"
I20260810 00:54:27.294430 29721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:27.319921 29721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:27.324901 29721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:27.325245 29721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:27.340180 29721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:54:27.344568 29721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:54:27.344666 29721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:27.344871 29721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:54:27.345000 29721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:27.448591 29721 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:36479
I20260810 00:54:27.448611 29931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:36479 every 8 connection(s)
I20260810 00:54:27.450280 29721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:54:27.459558 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 29721
I20260810 00:54:27.459775 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:54:27.460611 10629 external_mini_cluster.cc:1428] Setting key ab6cdf4846b8c351803e55d8dc67875a
Aug 10 00:54:27 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323267, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:54:27.478385 29936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:27.454589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:46877 (local address 127.10.97.65:58849)
0810 00:54:27.455472 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:27.455559 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:27.458129 (+ 2570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:27.459002 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:27.459014 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:27.459663 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:27.460531 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:27.460557 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:27.462091 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:27.462096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:27.464228 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:27.464243 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:27.465504 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:27.465512 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:27.465788 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:27.466606 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:27.466636 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:27.469462 (+ 2826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:27.473744 (+ 4282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:27.473752 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:27.473768 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:27.474367 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:27.474844 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:27.474849 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:27.474853 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:27.475114 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:27.475848 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:27.475857 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:27.476255 (+ 398us) client_negotiation.cc:769] Sending connection context
0810 00:54:27.476536 (+ 281us) client_negotiation.cc:240] Negotiation successful
0810 00:54:27.477047 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":334,"threads_started":1}
I20260810 00:54:27.480252 29935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:27.453405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:58849 (local address 127.10.97.126:46877)
0810 00:54:27.453940 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:27.453947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:27.455692 (+ 1745us) server_negotiation.cc:407] Connection header received
0810 00:54:27.458480 (+ 2788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:27.458487 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:27.458592 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:27.458731 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:27.460739 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:27.461874 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:27.464398 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:27.465048 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:27.465151 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:27.469734 (+ 4583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:27.469740 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:27.469745 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:27.469795 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:27.473552 (+ 3757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:27.474511 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:27.474516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:27.474520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:27.474687 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:27.475265 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:27.475270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:27.475274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:27.475660 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:27.479465 (+ 3805us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:27.479608 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 00:54:27.479887 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":1462,"threads_started":1}
I20260810 00:54:27.481961 29932 heartbeater.cc:347] Connected to a master server at 127.10.97.126:46877
I20260810 00:54:27.482321 29932 heartbeater.cc:464] Registering TS with master...
I20260810 00:54:27.483191 29932 heartbeater.cc:511] Master 127.10.97.126:46877 requested a full tablet report, sending...
I20260810 00:54:27.485679 29597 ts_manager.cc:194] Registered new tserver with Master: 843b48a518084bd4bfeada56d9f0a671 (127.10.97.65:36479)
I20260810 00:54:27.487634 29597 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:58849
WARNING: no policy specified for kudu/127.10.97.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
I20260810 00:54:27.528700 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.97.84:40887/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:46877
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:42349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 00:54:27 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323267, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
Aug 10 00:54:27 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323267, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
W20260810 00:54:28.023880 29940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:28.024408 29940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:28.024614 29940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:28.035928 29940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:28.036123 29940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:28.036387 29940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:54:28.051731 29940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:42349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.97.84:40887/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:46877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:28.053678 29940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:28.055446 29940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:28.096575 29951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:28.102605 29950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:28.104666 29955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:28.105178 29940 server_base.cc:1034] running on GCE node
I20260810 00:54:28.105955 29940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:28.110916 29940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:28.112238 29940 hybrid_clock.cc:648] HybridClock initialized: now 1786323268112134 us; error 89 us; skew 500 ppm
I20260810 00:54:28.112679 29940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:28 dist-test-slave-jlzn krb5kdc[28864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323268, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:28.116874 29940 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:54:28.122570 29940 webserver.cc:460] Webserver started at http://127.10.97.66:40677/ using document root <none> and password file <none>
I20260810 00:54:28.123481 29940 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:28.123621 29940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:28.124110 29940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 00:54:28 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323268, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
I20260810 00:54:28.156833 29940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "0a60731ddf814efda7c9022f2cca97ac"
format_stamp: "Formatted at 2026-08-10 00:54:28 on dist-test-slave-jlzn"
server_key: "2d7ec8807d7d544e9aabd69d162b2413"
server_key_iv: "69f5b5ddb7c6551ea65eb14c37846124"
server_key_version: "kuduclusterkey@0"
I20260810 00:54:28.163290 29940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "0a60731ddf814efda7c9022f2cca97ac"
format_stamp: "Formatted at 2026-08-10 00:54:28 on dist-test-slave-jlzn"
server_key: "2d7ec8807d7d544e9aabd69d162b2413"
server_key_iv: "69f5b5ddb7c6551ea65eb14c37846124"
server_key_version: "kuduclusterkey@0"
I20260810 00:54:28.190783 29940 fs_manager.cc:691] Time spent creating directory manager: real 0.027s user 0.006s sys 0.011s
I20260810 00:54:28.229918 29962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:28.231894 29940 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.014s sys 0.000s
I20260810 00:54:28.232187 29940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "0a60731ddf814efda7c9022f2cca97ac"
format_stamp: "Formatted at 2026-08-10 00:54:28 on dist-test-slave-jlzn"
server_key: "2d7ec8807d7d544e9aabd69d162b2413"
server_key_iv: "69f5b5ddb7c6551ea65eb14c37846124"
server_key_version: "kuduclusterkey@0"
I20260810 00:54:28.232540 29940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:28.284765 29940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:28.289975 29940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:28.290359 29940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:28.291569 29940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:54:28.293340 29940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:54:28.293432 29940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:28.293604 29940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:54:28.293670 29940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:28.345774 29940 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:42267
I20260810 00:54:28.345796 30155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:42267 every 8 connection(s)
I20260810 00:54:28.347487 29940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:54:28.355736 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 29940
I20260810 00:54:28.355926 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:54:28.356501 10629 external_mini_cluster.cc:1428] Setting key 0754e2aa57577e64b081fcb73c010e39
WARNING: no policy specified for kudu/127.10.97.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.67@KRBTEST.COM" created.
Aug 10 00:54:28 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323268, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:54:28.404767 30158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:28.350406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:35055 (local address 127.10.97.126:46877)
0810 00:54:28.370618 (+ 20212us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:28.370625 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:28.371783 (+ 1158us) server_negotiation.cc:407] Connection header received
0810 00:54:28.373903 (+ 2120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:28.373909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:28.373994 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:28.374091 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:28.384126 (+ 10035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:28.385298 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:28.387573 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:28.388090 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:28.388207 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:28.396552 (+ 8345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:28.396558 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:28.396562 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:28.396639 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:28.400193 (+ 3554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:28.401955 (+ 1762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:28.401961 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:28.401965 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:28.402111 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:28.402642 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:28.402648 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:28.402651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:28.403034 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:28.403172 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:28.404174 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0810 00:54:28.404401 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20079,"thread_start_us":21395,"threads_started":1}
I20260810 00:54:28.407215 30159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:28.351002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:46877 (local address 127.10.97.66:35055)
0810 00:54:28.371197 (+ 20195us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:28.371250 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:28.373615 (+ 2365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:28.382335 (+ 8720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:28.382348 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:28.382987 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:28.383897 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:28.383927 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:28.385531 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:28.385536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:28.387426 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:28.387439 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:28.388333 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:28.388341 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:28.388560 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:28.389279 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:28.389303 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:28.396299 (+ 6996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:28.400384 (+ 4085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:28.400393 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:28.400410 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:28.400981 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:28.402265 (+ 1284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:28.402270 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:28.402274 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:28.402511 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:28.403264 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:28.403271 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:28.403691 (+ 420us) client_negotiation.cc:769] Sending connection context
0810 00:54:28.405302 (+ 1611us) client_negotiation.cc:240] Negotiation successful
0810 00:54:28.405857 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19889,"thread_start_us":19659,"threads_started":1}
I20260810 00:54:28.409179 30156 heartbeater.cc:347] Connected to a master server at 127.10.97.126:46877
I20260810 00:54:28.409489 30156 heartbeater.cc:464] Registering TS with master...
I20260810 00:54:28.410379 30156 heartbeater.cc:511] Master 127.10.97.126:46877 requested a full tablet report, sending...
I20260810 00:54:28.412801 29597 ts_manager.cc:194] Registered new tserver with Master: 0a60731ddf814efda7c9022f2cca97ac (127.10.97.66:42267)
I20260810 00:54:28.413923 29597 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:35055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
I20260810 00:54:28.424463 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.97.84:40887/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:46877
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:42349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:54:28.491080 29932 heartbeater.cc:503] Master 127.10.97.126:46877 was elected leader, sending a full tablet report...
W20260810 00:54:28.748795 30162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:28.749090 30162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:28.749226 30162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:28.756098 30162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:28.756202 30162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:28.756350 30162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:54:28.765857 30162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:42349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.97.84:40887/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:46877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:28.766988 30162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:28.768229 30162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:28.780393 30176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:28.780431 30178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:28.780658 30174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:28.781351 30162 server_base.cc:1034] running on GCE node
I20260810 00:54:28.781890 30162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:28.782717 30162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:28.783885 30162 hybrid_clock.cc:648] HybridClock initialized: now 1786323268783846 us; error 27 us; skew 500 ppm
I20260810 00:54:28.784175 30162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:28 dist-test-slave-jlzn krb5kdc[28864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323268, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:28.787048 30162 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:54:28.788332 30162 webserver.cc:460] Webserver started at http://127.10.97.67:43279/ using document root <none> and password file <none>
I20260810 00:54:28.788932 30162 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:28.789053 30162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:28.789338 30162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 00:54:28 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323268, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
I20260810 00:54:28.807410 30162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "0480e933b2b940779e68356e9f76d962"
format_stamp: "Formatted at 2026-08-10 00:54:28 on dist-test-slave-jlzn"
server_key: "e75a1da48dcb9fc75fe6b742a18b7e75"
server_key_iv: "841a2e0e4a30c934ae0a59b7b962b6c3"
server_key_version: "kuduclusterkey@0"
I20260810 00:54:28.808506 30162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "0480e933b2b940779e68356e9f76d962"
format_stamp: "Formatted at 2026-08-10 00:54:28 on dist-test-slave-jlzn"
server_key: "e75a1da48dcb9fc75fe6b742a18b7e75"
server_key_iv: "841a2e0e4a30c934ae0a59b7b962b6c3"
server_key_version: "kuduclusterkey@0"
I20260810 00:54:28.815232 30162 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260810 00:54:28.829571 30189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:28.831089 30162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 00:54:28.831249 30162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "0480e933b2b940779e68356e9f76d962"
format_stamp: "Formatted at 2026-08-10 00:54:28 on dist-test-slave-jlzn"
server_key: "e75a1da48dcb9fc75fe6b742a18b7e75"
server_key_iv: "841a2e0e4a30c934ae0a59b7b962b6c3"
server_key_version: "kuduclusterkey@0"
I20260810 00:54:28.831444 30162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:28.846798 30162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:28.850524 30162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:28.850785 30162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:28.851719 30162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:54:28.853078 30162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:54:28.853158 30162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:28.853272 30162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:54:28.853328 30162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:28.904723 30162 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:42325
I20260810 00:54:28.904757 30361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:42325 every 8 connection(s)
I20260810 00:54:28.906283 30162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:54:28.913583 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 30162
I20260810 00:54:28.913740 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:54:28.914168 10629 external_mini_cluster.cc:1428] Setting key cd70378ea7e1b5ed75cc9d688ba1545f
Aug 10 00:54:28 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323268, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:54:28.925074 30365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:28.908684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:56047 (local address 127.10.97.126:46877)
0810 00:54:28.909109 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:28.909113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:28.909977 (+ 864us) server_negotiation.cc:407] Connection header received
0810 00:54:28.911739 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:28.911746 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:28.911890 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:28.911959 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:28.913458 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:28.914218 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:28.916200 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:28.916662 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:28.916768 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:28.920009 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:28.920013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:28.920017 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:28.920049 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:28.922250 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:28.922913 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:28.922918 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:28.922922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:28.923048 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:28.923459 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:28.923464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:28.923468 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:28.923738 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:28.923918 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:28.924504 (+ 586us) server_negotiation.cc:299] Negotiation successful
0810 00:54:28.924733 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":207,"threads_started":1}
I20260810 00:54:28.925869 30366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:28.909143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:46877 (local address 127.10.97.67:56047)
0810 00:54:28.909830 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:28.909869 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:28.911466 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:28.912311 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:28.912319 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:28.912720 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:28.913295 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:28.913313 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:28.914395 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:28.914401 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:28.916069 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:28.916083 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:28.916894 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:28.916903 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:28.917117 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:28.917784 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:28.917806 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:28.919784 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:28.922374 (+ 2590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:28.922383 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:28.922397 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:28.922803 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:28.923156 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:28.923161 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:28.923165 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:28.923355 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:28.923883 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:28.923890 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:28.924181 (+ 291us) client_negotiation.cc:769] Sending connection context
0810 00:54:28.924404 (+ 223us) client_negotiation.cc:240] Negotiation successful
0810 00:54:28.924846 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":275,"threads_started":1}
I20260810 00:54:28.926970 30363 heartbeater.cc:347] Connected to a master server at 127.10.97.126:46877
I20260810 00:54:28.927209 30363 heartbeater.cc:464] Registering TS with master...
I20260810 00:54:28.927908 30363 heartbeater.cc:511] Master 127.10.97.126:46877 requested a full tablet report, sending...
I20260810 00:54:28.929414 29595 ts_manager.cc:194] Registered new tserver with Master: 0480e933b2b940779e68356e9f76d962 (127.10.97.67:42325)
I20260810 00:54:28.930265 29595 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:56047
I20260810 00:54:28.941143 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:54:28.942807 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 29721
Aug 10 00:54:28 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:54:28.957950 30368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:28.943650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34838 (local address 127.10.97.65:36479)
0810 00:54:28.944262 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:28.944268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:28.944291 (+ 23us) server_negotiation.cc:407] Connection header received
0810 00:54:28.944343 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:28.944348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:28.944509 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:28.944622 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:28.945922 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:28.946908 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:28.948325 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:28.948827 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:28.948917 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:28.952735 (+ 3818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:28.952745 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:28.952756 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:28.952791 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:28.955319 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:28.955979 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:28.955985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:28.955987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:28.956145 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:28.956493 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:28.956496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:28.956498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:28.956801 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:28.957081 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:28.957417 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 00:54:28.957659 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":313,"threads_started":1}
I20260810 00:54:28.959124 29926 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:29.042208 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 29721
I20260810 00:54:29.063602 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 29940
Aug 10 00:54:29 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.66@KRBTEST.COM
I20260810 00:54:29.078465 30374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:29.064426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43032 (local address 127.10.97.66:42267)
0810 00:54:29.064941 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:29.064948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:29.064974 (+ 26us) server_negotiation.cc:407] Connection header received
0810 00:54:29.065035 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:29.065042 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:29.065268 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:29.065399 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:29.066527 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:29.067510 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:29.068793 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:29.069346 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:29.069440 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:29.073427 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:29.073439 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:29.073453 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:29.073485 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:29.076177 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:29.076700 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:29.076706 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:29.076708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:29.076844 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:29.077110 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:29.077113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:29.077115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:29.077442 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:29.077674 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:29.077854 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 00:54:29.078073 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":250,"threads_started":1}
I20260810 00:54:29.079681 30149 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:29.158471 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 29940
I20260810 00:54:29.183250 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 30162
Aug 10 00:54:29 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.67@KRBTEST.COM
I20260810 00:54:29.198619 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:29.184247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46002 (local address 127.10.97.67:42325)
0810 00:54:29.184896 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:29.184903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:29.184932 (+ 29us) server_negotiation.cc:407] Connection header received
0810 00:54:29.184991 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:29.184998 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:29.185221 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:29.185342 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:29.186584 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:29.187600 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:29.188938 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:29.189479 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:29.189571 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:29.193413 (+ 3842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:29.193423 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:29.193437 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:29.193471 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:29.196316 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:29.196929 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:29.196936 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:29.196938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:29.197066 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:29.197387 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:29.197390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:29.197392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:29.197712 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:29.197942 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:29.198155 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 00:54:29.198342 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":353,"threads_started":1}
I20260810 00:54:29.199875 30356 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:29.279072 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 30162
I20260810 00:54:29.298138 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 29545
I20260810 00:54:29.299324 29646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:29.402354 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 29545
I20260810 00:54:29.434928 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:46877
--webserver_interface=127.10.97.126
--webserver_port=44391
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:42349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:46877
--ranger_config_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.97.84:40887/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.97.84:40887/kms with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:54:29.750921 30379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:29.751168 30379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:29.751286 30379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:29.758074 30379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:54:29.758168 30379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:29.758240 30379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:29.758308 30379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:54:29.758366 30379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:54:29.767791 30379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:42349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.97.84:40887/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:46877
--ranger_config_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:46877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=44391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:29.768982 30379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:29.770190 30379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:29.780892 30384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:29.780942 30385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:29.780969 30387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:29.781695 30379 server_base.cc:1034] running on GCE node
I20260810 00:54:29.782492 30379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:29.784029 30379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:29.785247 30379 hybrid_clock.cc:648] HybridClock initialized: now 1786323269785188 us; error 48 us; skew 500 ppm
I20260810 00:54:29.785604 30379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:29 dist-test-slave-jlzn krb5kdc[28864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323269, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:29.791463 30379 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:54:29.792761 30379 webserver.cc:460] Webserver started at http://127.10.97.126:44391/ using document root <none> and password file <none>
I20260810 00:54:29.793324 30379 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:29.793443 30379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 00:54:29 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323269, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
I20260810 00:54:29.839172 30379 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 00:54:29.842790 30404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:29.844369 30379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 00:54:29.844548 30379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "118001e1cd7943468314202ab030828a"
format_stamp: "Formatted at 2026-08-10 00:54:25 on dist-test-slave-jlzn"
server_key: "104c605cb116f56df638f0efde29e343"
server_key_iv: "79290a8ed0176a15184e1e017474c47c"
server_key_version: "kuduclusterkey@0"
I20260810 00:54:29.845815 30379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:29.865309 30379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:29.868544 30379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:29.868767 30379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:29.885483 30379 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:46877
I20260810 00:54:29.885531 30480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:46877 every 8 connection(s)
I20260810 00:54:29.886823 30379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:54:29.890775 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 30379
I20260810 00:54:29.891871 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.97.84:40887/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:36479
--local_ip_for_outbound_sockets=127.10.97.65
--tserver_master_addrs=127.10.97.126:46877
--webserver_port=38415
--webserver_interface=127.10.97.65
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:42349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.97.84:40887/kms with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:54:29.894413 30481 sys_catalog.cc:263] Verifying existing consensus state
I20260810 00:54:29.898638 30481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a: Bootstrap starting.
I20260810 00:54:29.917368 30481 log.cc:824] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a: Log is configured to *not* fsync() on all Append() calls
I20260810 00:54:29.925872 30481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 00:54:29.926362 30481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a: Bootstrap complete.
I20260810 00:54:29.934970 30481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118001e1cd7943468314202ab030828a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 46877 } }
I20260810 00:54:29.935766 30481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 118001e1cd7943468314202ab030828a, State: Initialized, Role: FOLLOWER
I20260810 00:54:29.936399 30481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118001e1cd7943468314202ab030828a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 46877 } }
I20260810 00:54:29.936628 30481 raft_consensus.cc:388] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:54:29.936739 30481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:54:29.936903 30481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:54:29.940873 30481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118001e1cd7943468314202ab030828a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 46877 } }
I20260810 00:54:29.941334 30481 leader_election.cc:302] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 118001e1cd7943468314202ab030828a; no voters:
I20260810 00:54:29.941938 30481 leader_election.cc:288] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 00:54:29.942099 30489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [term 2 FOLLOWER]: Leader election won for term 2
I20260810 00:54:29.943209 30489 raft_consensus.cc:686] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [term 2 LEADER]: Becoming Leader. State: Replica: 118001e1cd7943468314202ab030828a, State: Running, Role: LEADER
I20260810 00:54:29.943848 30489 consensus_queue.cc:237] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118001e1cd7943468314202ab030828a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 46877 } }
I20260810 00:54:29.944111 30481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:54:29.946341 30491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 118001e1cd7943468314202ab030828a. Latest consensus state: current_term: 2 leader_uuid: "118001e1cd7943468314202ab030828a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118001e1cd7943468314202ab030828a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 46877 } } }
I20260810 00:54:29.946729 30491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [sys.catalog]: This master's current role is: LEADER
I20260810 00:54:29.947244 30490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "118001e1cd7943468314202ab030828a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118001e1cd7943468314202ab030828a" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 46877 } } }
I20260810 00:54:29.947526 30490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a [sys.catalog]: This master's current role is: LEADER
I20260810 00:54:29.957568 30496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:54:29.963167 30496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:54:29.964576 30496 catalog_manager.cc:1259] Loaded cluster ID: 580e0988b14d4b56b30868fd6641da88
I20260810 00:54:29.964741 30496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:54:29.968678 30496 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:54:29.970088 30496 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a: Loaded TSK: 0
I20260810 00:54:29.971571 30496 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:54:29.976202 30481 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260810 00:54:30.341409 30484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:30.341830 30484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:30.342020 30484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:30.353303 30484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:30.353441 30484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:30.353675 30484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:54:30.368830 30484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:42349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.97.84:40887/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:36479
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=38415
--tserver_master_addrs=127.10.97.126:46877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:30.370744 30484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:30.372625 30484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:30.390899 30534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:30.393363 30539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:30.400171 30484 server_base.cc:1034] running on GCE node
W20260810 00:54:30.400434 30541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:30.401139 30484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:30.402205 30484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:30.403482 30484 hybrid_clock.cc:648] HybridClock initialized: now 1786323270403418 us; error 50 us; skew 500 ppm
I20260810 00:54:30.403867 30484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:30 dist-test-slave-jlzn krb5kdc[28864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323270, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:30.407193 30484 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:54:30.408914 30484 webserver.cc:460] Webserver started at http://127.10.97.65:38415/ using document root <none> and password file <none>
I20260810 00:54:30.409590 30484 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:30.409725 30484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 00:54:30 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323270, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
I20260810 00:54:30.441747 30484 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 00:54:30.445264 30552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:30.446930 30484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 00:54:30.447094 30484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "843b48a518084bd4bfeada56d9f0a671"
format_stamp: "Formatted at 2026-08-10 00:54:27 on dist-test-slave-jlzn"
server_key: "8146f5626c92e97baa147ff2f64dad70"
server_key_iv: "23ab428dcb091a801960eeb2aedae8e1"
server_key_version: "kuduclusterkey@0"
I20260810 00:54:30.448487 30484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:30.482195 30484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:30.487124 30484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:30.487486 30484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:30.488768 30484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:54:30.490561 30484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:54:30.490701 30484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:30.490824 30484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:54:30.490911 30484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:30.557086 30484 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:36479
I20260810 00:54:30.557101 30711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:36479 every 8 connection(s)
I20260810 00:54:30.559002 30484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:54:30.562649 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 30484
I20260810 00:54:30.563536 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.97.84:40887/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:42267
--local_ip_for_outbound_sockets=127.10.97.66
--tserver_master_addrs=127.10.97.126:46877
--webserver_port=40677
--webserver_interface=127.10.97.66
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:42349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.97.84:40887/kms with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 00:54:30 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323270, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:54:30.599370 30717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:30.562549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:46877 (local address 127.10.97.65:41959)
0810 00:54:30.576751 (+ 14202us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:30.576804 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:30.579784 (+ 2980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:30.582718 (+ 2934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:30.582729 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:30.583322 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:30.584119 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:30.584145 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:30.586545 (+ 2400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:30.586550 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:30.587908 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:30.587920 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:30.588993 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:30.589001 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:30.589272 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:30.589887 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:30.589909 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:30.591930 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:30.595173 (+ 3243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:30.595182 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:30.595195 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:30.595586 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:30.596049 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:30.596054 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:30.596058 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:30.596258 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:30.597015 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:30.597023 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:30.597315 (+ 292us) client_negotiation.cc:769] Sending connection context
0810 00:54:30.597529 (+ 214us) client_negotiation.cc:240] Negotiation successful
0810 00:54:30.598100 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13904,"thread_start_us":14485,"threads_started":1}
I20260810 00:54:30.599352 30715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:30.562547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:41959 (local address 127.10.97.126:46877)
0810 00:54:30.574494 (+ 11947us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:30.574515 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:30.576939 (+ 2424us) server_negotiation.cc:407] Connection header received
0810 00:54:30.580487 (+ 3548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:30.580503 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:30.580921 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:30.581253 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:30.584364 (+ 3111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:30.586306 (+ 1942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:30.588040 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:30.588761 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:30.588944 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:30.592165 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:30.592172 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:30.592184 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:30.592228 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:30.595037 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:30.595738 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:30.595744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:30.595749 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:30.595943 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:30.596374 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:30.596379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:30.596383 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:30.596887 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:30.597072 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:30.597796 (+ 724us) server_negotiation.cc:299] Negotiation successful
0810 00:54:30.598155 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11329,"thread_start_us":11080,"threads_started":1}
I20260810 00:54:30.602982 30712 heartbeater.cc:347] Connected to a master server at 127.10.97.126:46877
I20260810 00:54:30.603302 30712 heartbeater.cc:464] Registering TS with master...
I20260810 00:54:30.604107 30712 heartbeater.cc:511] Master 127.10.97.126:46877 requested a full tablet report, sending...
W20260810 00:54:30.607082 30712 heartbeater.cc:644] Failed to heartbeat to 127.10.97.126:46877 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 00:54:30.607311 30712 heartbeater.cc:464] Registering TS with master...
I20260810 00:54:30.607565 30712 heartbeater.cc:511] Master 127.10.97.126:46877 requested a full tablet report, sending...
I20260810 00:54:30.608840 30712 heartbeater.cc:464] Registering TS with master...
I20260810 00:54:30.609091 30712 heartbeater.cc:511] Master 127.10.97.126:46877 requested a full tablet report, sending...
W20260810 00:54:30.610175 30712 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 00:54:30.993758 30718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:30.994078 30718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:30.994230 30718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:31.003861 30718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:31.003978 30718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:31.004153 30718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:54:31.014253 30718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:42349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.97.84:40887/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:42267
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=40677
--tserver_master_addrs=127.10.97.126:46877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:31.015869 30718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:31.017136 30718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:31.030969 30737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:31.031432 30739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:31.031075 30736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:31.032073 30718 server_base.cc:1034] running on GCE node
I20260810 00:54:31.032698 30718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:31.036094 30718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:31.037352 30718 hybrid_clock.cc:648] HybridClock initialized: now 1786323271037285 us; error 63 us; skew 500 ppm
I20260810 00:54:31.037642 30718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:31 dist-test-slave-jlzn krb5kdc[28864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323271, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:31.040789 30718 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:54:31.042392 30718 webserver.cc:460] Webserver started at http://127.10.97.66:40677/ using document root <none> and password file <none>
I20260810 00:54:31.043015 30718 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:31.043155 30718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 00:54:31 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323271, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
I20260810 00:54:31.078290 30718 fs_manager.cc:709] Time spent opening directory manager: real 0.020s user 0.004s sys 0.012s
I20260810 00:54:31.081941 30751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:31.083734 30718 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 00:54:31.083995 30718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "0a60731ddf814efda7c9022f2cca97ac"
format_stamp: "Formatted at 2026-08-10 00:54:28 on dist-test-slave-jlzn"
server_key: "2d7ec8807d7d544e9aabd69d162b2413"
server_key_iv: "69f5b5ddb7c6551ea65eb14c37846124"
server_key_version: "kuduclusterkey@0"
I20260810 00:54:31.085448 30718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:31.119624 30718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:31.124850 30718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:31.125241 30718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:31.126508 30718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:54:31.128291 30718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:54:31.128408 30718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:31.128516 30718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:54:31.128597 30718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:31.167562 30718 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:42267
I20260810 00:54:31.167642 30926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:42267 every 8 connection(s)
I20260810 00:54:31.169503 30718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:54:31.172605 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 30718
I20260810 00:54:31.173424 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.97.84:40887/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:42325
--local_ip_for_outbound_sockets=127.10.97.67
--tserver_master_addrs=127.10.97.126:46877
--webserver_port=43279
--webserver_interface=127.10.97.67
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:42349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.97.84:40887/kms with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 00:54:31 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323271, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
Aug 10 00:54:31 dist-test-slave-jlzn krb5kdc[28864](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786323271, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:31.203347 30932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:31.172807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:46877 (local address 127.10.97.66:52281)
0810 00:54:31.175429 (+ 2622us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:31.175488 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:31.177935 (+ 2447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:31.183623 (+ 5688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:31.183636 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:31.184287 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:31.185800 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:31.185826 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:31.187391 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:31.187396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:31.189430 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:31.189446 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:31.190626 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:31.190634 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:31.190980 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:31.191741 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:31.191771 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:31.194562 (+ 2791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:31.198643 (+ 4081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:31.198653 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:31.198671 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:31.199266 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:31.199701 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:31.199706 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:31.199710 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:31.199977 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:31.200685 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:31.200694 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:31.201101 (+ 407us) client_negotiation.cc:769] Sending connection context
0810 00:54:31.201396 (+ 295us) client_negotiation.cc:240] Negotiation successful
0810 00:54:31.201933 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2321,"thread_start_us":2102,"threads_started":1}
I20260810 00:54:31.204450 30934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:31.173393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:52281 (local address 127.10.97.126:46877)
0810 00:54:31.180169 (+ 6776us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:31.180175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:31.180200 (+ 25us) server_negotiation.cc:407] Connection header received
0810 00:54:31.182769 (+ 2569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:31.182774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:31.182858 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:31.182975 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:31.186023 (+ 3048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:31.187189 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:31.189599 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:31.190230 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:31.190348 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:31.194861 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:31.194869 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:31.194874 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:31.194918 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:31.198477 (+ 3559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:31.199407 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:31.199413 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:31.199417 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:31.199569 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:31.200108 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:31.200113 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:31.200117 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:31.200474 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:31.200612 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:31.203850 (+ 3238us) server_negotiation.cc:299] Negotiation successful
0810 00:54:31.204086 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6643,"thread_start_us":6543,"threads_started":1}
I20260810 00:54:31.206470 30927 heartbeater.cc:347] Connected to a master server at 127.10.97.126:46877
I20260810 00:54:31.206879 30927 heartbeater.cc:464] Registering TS with master...
I20260810 00:54:31.207736 30927 heartbeater.cc:511] Master 127.10.97.126:46877 requested a full tablet report, sending...
I20260810 00:54:31.209635 30434 ts_manager.cc:194] Registered new tserver with Master: 0a60731ddf814efda7c9022f2cca97ac (127.10.97.66:42267)
I20260810 00:54:31.211663 30434 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:52281
W20260810 00:54:31.576668 30935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:31.576933 30935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:31.577183 30935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:31.586809 30935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:31.586987 30935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:31.587162 30935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:54:31.598094 30935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:42349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.97.84:40887/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:42325
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=43279
--tserver_master_addrs=127.10.97.126:46877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:31.599278 30935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:31.600564 30935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 00:54:31.610318 30712 heartbeater.cc:464] Registering TS with master...
I20260810 00:54:31.610719 30712 heartbeater.cc:511] Master 127.10.97.126:46877 requested a full tablet report, sending...
I20260810 00:54:31.612056 30434 ts_manager.cc:194] Registered new tserver with Master: 843b48a518084bd4bfeada56d9f0a671 (127.10.97.65:36479)
I20260810 00:54:31.613289 30434 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:41959
W20260810 00:54:31.621336 30943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:31.621974 30944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:31.629778 30947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:31.630609 30935 server_base.cc:1034] running on GCE node
I20260810 00:54:31.631237 30935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:31.632333 30935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:31.633590 30935 hybrid_clock.cc:648] HybridClock initialized: now 1786323271633527 us; error 58 us; skew 500 ppm
I20260810 00:54:31.633903 30935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:31 dist-test-slave-jlzn krb5kdc[28864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323271, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:31.636945 30935 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:54:31.648305 30935 webserver.cc:460] Webserver started at http://127.10.97.67:43279/ using document root <none> and password file <none>
I20260810 00:54:31.649216 30935 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:31.649376 30935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 00:54:31 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323271, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
Aug 10 00:54:31 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323271, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
I20260810 00:54:31.705925 30935 fs_manager.cc:709] Time spent opening directory manager: real 0.016s user 0.007s sys 0.004s
I20260810 00:54:31.715677 30957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:31.717437 30935 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.003s sys 0.005s
I20260810 00:54:31.717622 30935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "0480e933b2b940779e68356e9f76d962"
format_stamp: "Formatted at 2026-08-10 00:54:28 on dist-test-slave-jlzn"
server_key: "e75a1da48dcb9fc75fe6b742a18b7e75"
server_key_iv: "841a2e0e4a30c934ae0a59b7b962b6c3"
server_key_version: "kuduclusterkey@0"
I20260810 00:54:31.719013 30935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:31.774770 30935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:31.779942 30935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:31.780313 30935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:31.781512 30935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:54:31.783262 30935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:54:31.783371 30935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:31.783473 30935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:54:31.783550 30935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:31.832875 30935 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:42325
I20260810 00:54:31.832960 31131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:42325 every 8 connection(s)
I20260810 00:54:31.834659 30935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:54:31.838949 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 30935
Aug 10 00:54:31 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323271, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:54:31.871384 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:31.839705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:55957 (local address 127.10.97.126:46877)
0810 00:54:31.847506 (+ 7801us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:31.847512 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:31.847531 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:54:31.847595 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:31.847600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:31.847704 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:31.847833 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:31.855688 (+ 7855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:31.856895 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:31.859387 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:31.860072 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:31.860189 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:31.864417 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:31.864424 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:31.864428 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:31.864486 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:31.867738 (+ 3252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:31.868619 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:31.868623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:31.868627 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:31.868770 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:31.869306 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:31.869312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:31.869315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:31.869638 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:31.869754 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:31.870825 (+ 1071us) server_negotiation.cc:299] Negotiation successful
0810 00:54:31.871040 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7691,"thread_start_us":7598,"threads_started":1}
I20260810 00:54:31.873512 31141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:31.838072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:46877 (local address 127.10.97.67:55957)
0810 00:54:31.844336 (+ 6264us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:31.844380 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:31.846847 (+ 2467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:31.853682 (+ 6835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:31.853696 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:31.854542 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:31.855446 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:31.855471 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:31.857143 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:31.857148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:31.859210 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:31.859226 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:31.860479 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:31.860485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:31.860746 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:31.861585 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:31.861618 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:31.864156 (+ 2538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:31.867914 (+ 3758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:31.867923 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:31.867937 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:31.868473 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:31.868895 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:31.868899 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:31.868903 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:31.869173 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:31.869832 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:31.869839 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:31.870229 (+ 390us) client_negotiation.cc:769] Sending connection context
0810 00:54:31.870503 (+ 274us) client_negotiation.cc:240] Negotiation successful
0810 00:54:31.872214 (+ 1711us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5976,"thread_start_us":5755,"threads_started":1}
I20260810 00:54:31.875322 31134 heartbeater.cc:347] Connected to a master server at 127.10.97.126:46877
I20260810 00:54:31.875635 31134 heartbeater.cc:464] Registering TS with master...
I20260810 00:54:31.876505 31134 heartbeater.cc:511] Master 127.10.97.126:46877 requested a full tablet report, sending...
I20260810 00:54:31.878185 30434 ts_manager.cc:194] Registered new tserver with Master: 0480e933b2b940779e68356e9f76d962 (127.10.97.67:42325)
I20260810 00:54:31.879264 30434 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:55957
I20260810 00:54:31.879736 31147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:31.847463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58768 (local address 127.10.97.126:46877)
0810 00:54:31.861830 (+ 14367us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:31.861835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:31.861852 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:54:31.861901 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:31.861906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:31.861980 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:31.862091 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:31.864337 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:31.865872 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:31.868192 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:31.868928 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:31.869048 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:31.872558 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:31.872565 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:31.872570 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:31.872604 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:31.875947 (+ 3343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:31.876766 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:31.876771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:31.876774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:31.876923 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:31.877417 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:31.877423 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:31.877426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:31.877749 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:31.878041 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:31.879156 (+ 1115us) server_negotiation.cc:299] Negotiation successful
0810 00:54:31.879372 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14229,"thread_start_us":14278,"threads_started":1}
I20260810 00:54:31.882638 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:54:31.912679 31147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:31.889562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58772 (local address 127.10.97.126:46877)
0810 00:54:31.889711 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:31.889717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:31.894911 (+ 5194us) server_negotiation.cc:407] Connection header received
0810 00:54:31.895074 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:31.895080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:31.895150 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:31.895252 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:31.897275 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:31.898311 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:31.900219 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:31.900864 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:31.901001 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:31.903896 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:31.903902 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:31.903907 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:31.903942 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:31.907125 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:31.907891 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:31.907897 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:31.907900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:31.908047 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:31.908437 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:31.908441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:31.908444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:31.908796 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:31.911992 (+ 3196us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:31.912126 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 00:54:31.912344 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 00:54:31.918394 30434 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:54:31.929157 30434 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 10 00:54:31 dist-test-slave-jlzn krb5kdc[28864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323271, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for HTTP/127.10.97.84@KRBTEST.COM
I20260810 00:54:32.010895 31164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:31.967143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:42267 (local address 127.0.0.1:38480)
0810 00:54:32.001738 (+ 34595us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:32.001756 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:32.001884 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:32.002800 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:32.002808 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:32.002840 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:54:32.003282 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:32.003298 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.005328 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.005333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:32.007688 (+ 2355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:32.007703 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.009798 (+ 2095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.009805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:32.010046 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:32.010105 (+ 59us) client_negotiation.cc:769] Sending connection context
0810 00:54:32.010238 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 00:54:32.010416 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34461,"mutex_wait_us":2,"thread_start_us":33336,"threads_started":1}
I20260810 00:54:32.012055 31169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:31.967659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38480 (local address 127.10.97.66:42267)
0810 00:54:32.000291 (+ 32632us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:32.000302 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:32.002005 (+ 1703us) server_negotiation.cc:407] Connection header received
0810 00:54:32.002118 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:32.002125 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:32.002332 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:32.002505 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:54:32.003509 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.005154 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.007905 (+ 2751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.009295 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.009427 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:32.009624 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:32.011485 (+ 1861us) server_negotiation.cc:299] Negotiation successful
0810 00:54:32.011682 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32441,"thread_start_us":32303,"threads_started":1}
I20260810 00:54:32.016232 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:31.960350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42946 (local address 127.10.97.67:42325)
0810 00:54:31.968277 (+ 7927us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:31.968286 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:32.001065 (+ 32779us) server_negotiation.cc:407] Connection header received
0810 00:54:32.001396 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:32.001404 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:32.001673 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:32.001890 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:54:32.007121 (+ 5231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.008760 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.013264 (+ 4504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.014611 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.014703 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:32.014928 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:32.015677 (+ 749us) server_negotiation.cc:299] Negotiation successful
0810 00:54:32.015885 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7740,"thread_start_us":7596,"threads_started":1}
I20260810 00:54:32.017134 31168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:31.959708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:42325 (local address 127.0.0.1:42946)
0810 00:54:32.000894 (+ 41186us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:32.000940 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:32.001167 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:32.006552 (+ 5385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:32.006557 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:32.006590 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:54:32.006952 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:32.006963 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.009108 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.009112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:32.011037 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:32.011050 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.015139 (+ 4089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.015145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:32.015309 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:32.015340 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 00:54:32.016640 (+ 1300us) client_negotiation.cc:240] Negotiation successful
0810 00:54:32.016793 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40897,"mutex_wait_us":1,"thread_start_us":41920,"threads_started":1}
I20260810 00:54:32.017381 30828 tablet_service.cc:1467] Processing CreateTablet for tablet 3b2ec717e5374c0cabb1fb657e63efdf (DEFAULT_TABLE table=test-table [id=d9c7cedaccf042398964b05693b47508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:54:32.018869 30828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b2ec717e5374c0cabb1fb657e63efdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:54:32.020740 31031 tablet_service.cc:1467] Processing CreateTablet for tablet 3b2ec717e5374c0cabb1fb657e63efdf (DEFAULT_TABLE table=test-table [id=d9c7cedaccf042398964b05693b47508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:54:32.022226 31031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b2ec717e5374c0cabb1fb657e63efdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:54:32.023797 31170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:32.001484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:36479 (local address 127.0.0.1:43340)
0810 00:54:32.003604 (+ 2120us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:32.003622 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:32.003736 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:32.016381 (+ 12645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:32.016386 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:32.016409 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:54:32.016739 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:32.016752 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.018539 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.018544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:32.021158 (+ 2614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:32.021171 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.023065 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.023071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:32.023278 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:32.023308 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 00:54:32.023382 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 00:54:32.023494 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2004,"thread_start_us":2330,"threads_started":1}
I20260810 00:54:32.024593 31172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:32.002599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43340 (local address 127.10.97.65:36479)
0810 00:54:32.015583 (+ 12984us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:32.015591 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:32.015615 (+ 24us) server_negotiation.cc:407] Connection header received
0810 00:54:32.015708 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:32.015715 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:32.015954 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:32.016129 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:54:32.016950 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.018413 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.021317 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.022602 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.022724 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:32.022930 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:32.024054 (+ 1124us) server_negotiation.cc:299] Negotiation successful
0810 00:54:32.024212 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12808,"thread_start_us":12804,"threads_started":1}
I20260810 00:54:32.028131 30623 tablet_service.cc:1467] Processing CreateTablet for tablet 3b2ec717e5374c0cabb1fb657e63efdf (DEFAULT_TABLE table=test-table [id=d9c7cedaccf042398964b05693b47508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:54:32.029443 30623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b2ec717e5374c0cabb1fb657e63efdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:54:32.055295 31174 tablet_bootstrap.cc:492] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671: Bootstrap starting.
I20260810 00:54:32.058910 31174 tablet_bootstrap.cc:654] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671: Neither blocks nor log segments found. Creating new log.
I20260810 00:54:32.060272 31174 log.cc:824] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671: Log is configured to *not* fsync() on all Append() calls
I20260810 00:54:32.070446 31174 tablet_bootstrap.cc:492] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671: No bootstrap required, opened a new log
I20260810 00:54:32.070809 31174 ts_tablet_manager.cc:1397] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260810 00:54:32.072243 31173 tablet_bootstrap.cc:492] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac: Bootstrap starting.
I20260810 00:54:32.073853 31177 tablet_bootstrap.cc:492] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962: Bootstrap starting.
I20260810 00:54:32.078227 31177 tablet_bootstrap.cc:654] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962: Neither blocks nor log segments found. Creating new log.
I20260810 00:54:32.079457 31177 log.cc:824] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962: Log is configured to *not* fsync() on all Append() calls
I20260810 00:54:32.082047 31173 tablet_bootstrap.cc:654] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac: Neither blocks nor log segments found. Creating new log.
I20260810 00:54:32.082894 31174 raft_consensus.cc:348] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0480e933b2b940779e68356e9f76d962" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 42325 } } peers { permanent_uuid: "0a60731ddf814efda7c9022f2cca97ac" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42267 } } peers { permanent_uuid: "843b48a518084bd4bfeada56d9f0a671" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36479 } }
I20260810 00:54:32.083420 31174 raft_consensus.cc:374] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:54:32.083576 31174 raft_consensus.cc:729] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843b48a518084bd4bfeada56d9f0a671, State: Initialized, Role: FOLLOWER
I20260810 00:54:32.084426 31174 consensus_queue.cc:260] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0480e933b2b940779e68356e9f76d962" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 42325 } } peers { permanent_uuid: "0a60731ddf814efda7c9022f2cca97ac" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42267 } } peers { permanent_uuid: "843b48a518084bd4bfeada56d9f0a671" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36479 } }
I20260810 00:54:32.089804 31173 log.cc:824] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac: Log is configured to *not* fsync() on all Append() calls
I20260810 00:54:32.091650 30712 heartbeater.cc:503] Master 127.10.97.126:46877 was elected leader, sending a full tablet report...
I20260810 00:54:32.092474 31174 ts_tablet_manager.cc:1428] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671: Time spent starting tablet: real 0.021s user 0.017s sys 0.001s
I20260810 00:54:32.106503 31177 tablet_bootstrap.cc:492] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962: No bootstrap required, opened a new log
I20260810 00:54:32.106873 31177 ts_tablet_manager.cc:1397] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962: Time spent bootstrapping tablet: real 0.033s user 0.008s sys 0.004s
I20260810 00:54:32.113379 31173 tablet_bootstrap.cc:492] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac: No bootstrap required, opened a new log
I20260810 00:54:32.113724 31173 ts_tablet_manager.cc:1397] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac: Time spent bootstrapping tablet: real 0.042s user 0.009s sys 0.002s
I20260810 00:54:32.117172 31177 raft_consensus.cc:348] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0480e933b2b940779e68356e9f76d962" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 42325 } } peers { permanent_uuid: "0a60731ddf814efda7c9022f2cca97ac" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42267 } } peers { permanent_uuid: "843b48a518084bd4bfeada56d9f0a671" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36479 } }
I20260810 00:54:32.117686 31177 raft_consensus.cc:374] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:54:32.117844 31177 raft_consensus.cc:729] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0480e933b2b940779e68356e9f76d962, State: Initialized, Role: FOLLOWER
I20260810 00:54:32.118616 31177 consensus_queue.cc:260] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0480e933b2b940779e68356e9f76d962" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 42325 } } peers { permanent_uuid: "0a60731ddf814efda7c9022f2cca97ac" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42267 } } peers { permanent_uuid: "843b48a518084bd4bfeada56d9f0a671" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36479 } }
I20260810 00:54:32.127290 31134 heartbeater.cc:503] Master 127.10.97.126:46877 was elected leader, sending a full tablet report...
I20260810 00:54:32.128288 31177 ts_tablet_manager.cc:1428] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962: Time spent starting tablet: real 0.021s user 0.014s sys 0.004s
I20260810 00:54:32.131139 31173 raft_consensus.cc:348] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0480e933b2b940779e68356e9f76d962" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 42325 } } peers { permanent_uuid: "0a60731ddf814efda7c9022f2cca97ac" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42267 } } peers { permanent_uuid: "843b48a518084bd4bfeada56d9f0a671" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36479 } }
I20260810 00:54:32.131649 31173 raft_consensus.cc:374] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:54:32.131767 31173 raft_consensus.cc:729] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a60731ddf814efda7c9022f2cca97ac, State: Initialized, Role: FOLLOWER
I20260810 00:54:32.132470 31173 consensus_queue.cc:260] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0480e933b2b940779e68356e9f76d962" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 42325 } } peers { permanent_uuid: "0a60731ddf814efda7c9022f2cca97ac" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42267 } } peers { permanent_uuid: "843b48a518084bd4bfeada56d9f0a671" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36479 } }
I20260810 00:54:32.133622 30927 heartbeater.cc:503] Master 127.10.97.126:46877 was elected leader, sending a full tablet report...
I20260810 00:54:32.134317 31173 ts_tablet_manager.cc:1428] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac: Time spent starting tablet: real 0.020s user 0.006s sys 0.006s
W20260810 00:54:32.182919 30931 tablet.cc:2369] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:54:32.311748 30714 tablet.cc:2369] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:54:32.356463 31135 tablet.cc:2369] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:54:32.394217 31183 raft_consensus.cc:482] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:54:32.394466 31183 raft_consensus.cc:504] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0480e933b2b940779e68356e9f76d962" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 42325 } } peers { permanent_uuid: "0a60731ddf814efda7c9022f2cca97ac" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42267 } } peers { permanent_uuid: "843b48a518084bd4bfeada56d9f0a671" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36479 } }
I20260810 00:54:32.396082 31183 leader_election.cc:288] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a60731ddf814efda7c9022f2cca97ac (127.10.97.66:42267), 843b48a518084bd4bfeada56d9f0a671 (127.10.97.65:36479)
I20260810 00:54:32.402338 31194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:32.396272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:42267 (local address 127.10.97.67:45509)
0810 00:54:32.396722 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:32.396740 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:32.396871 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:32.397434 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:32.397437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:32.397462 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:54:32.397747 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:32.397760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.398912 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.398917 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:32.400394 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:32.400406 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.401646 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.401652 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:32.401822 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:32.401856 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 00:54:32.401953 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 00:54:32.402097 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":216,"threads_started":1}
I20260810 00:54:32.402354 31169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:32.396473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:45509 (local address 127.10.97.66:42267)
0810 00:54:32.396661 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:32.396666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:32.396811 (+ 145us) server_negotiation.cc:407] Connection header received
0810 00:54:32.397087 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:32.397093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:32.397175 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:32.397279 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:54:32.397904 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.398784 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.400549 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.401489 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.401606 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:32.401658 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:32.401977 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 00:54:32.402101 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 00:54:32.402786 31172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:32.397016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:54851 (local address 127.10.97.65:36479)
0810 00:54:32.397211 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:32.397216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:32.397504 (+ 288us) server_negotiation.cc:407] Connection header received
0810 00:54:32.397627 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:32.397633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:32.397704 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:32.397802 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:54:32.398382 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.399154 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.400993 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.401863 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.402008 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:32.402073 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:32.402446 (+ 373us) server_negotiation.cc:299] Negotiation successful
0810 00:54:32.402577 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 00:54:32.402815 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:32.396883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:36479 (local address 127.10.97.67:54851)
0810 00:54:32.397407 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:32.397420 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:32.397524 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:32.397814 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:32.397817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:32.397838 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:54:32.398219 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:32.398231 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.399290 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.399294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:32.400848 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:32.400863 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.402067 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.402073 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:32.402271 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:32.402301 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 00:54:32.402397 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 00:54:32.402513 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":267,"threads_started":1}
I20260810 00:54:32.403488 30856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2ec717e5374c0cabb1fb657e63efdf" candidate_uuid: "0480e933b2b940779e68356e9f76d962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a60731ddf814efda7c9022f2cca97ac" is_pre_election: true
I20260810 00:54:32.403846 30652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2ec717e5374c0cabb1fb657e63efdf" candidate_uuid: "0480e933b2b940779e68356e9f76d962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843b48a518084bd4bfeada56d9f0a671" is_pre_election: true
I20260810 00:54:32.404106 30856 raft_consensus.cc:2454] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0480e933b2b940779e68356e9f76d962 in term 0.
I20260810 00:54:32.404369 30652 raft_consensus.cc:2454] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0480e933b2b940779e68356e9f76d962 in term 0.
I20260810 00:54:32.404868 30960 leader_election.cc:302] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0480e933b2b940779e68356e9f76d962, 0a60731ddf814efda7c9022f2cca97ac; no voters:
I20260810 00:54:32.405222 31183 raft_consensus.cc:2781] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:54:32.405351 31183 raft_consensus.cc:482] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:54:32.405442 31183 raft_consensus.cc:3037] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:54:32.407527 31183 raft_consensus.cc:504] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0480e933b2b940779e68356e9f76d962" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 42325 } } peers { permanent_uuid: "0a60731ddf814efda7c9022f2cca97ac" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42267 } } peers { permanent_uuid: "843b48a518084bd4bfeada56d9f0a671" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36479 } }
I20260810 00:54:32.408085 31183 leader_election.cc:288] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [CANDIDATE]: Term 1 election: Requested vote from peers 0a60731ddf814efda7c9022f2cca97ac (127.10.97.66:42267), 843b48a518084bd4bfeada56d9f0a671 (127.10.97.65:36479)
I20260810 00:54:32.408473 30856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2ec717e5374c0cabb1fb657e63efdf" candidate_uuid: "0480e933b2b940779e68356e9f76d962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a60731ddf814efda7c9022f2cca97ac"
I20260810 00:54:32.408532 30652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2ec717e5374c0cabb1fb657e63efdf" candidate_uuid: "0480e933b2b940779e68356e9f76d962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843b48a518084bd4bfeada56d9f0a671"
I20260810 00:54:32.408640 30856 raft_consensus.cc:3037] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:54:32.408705 30652 raft_consensus.cc:3037] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:54:32.410900 30856 raft_consensus.cc:2454] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0480e933b2b940779e68356e9f76d962 in term 1.
I20260810 00:54:32.411325 30652 raft_consensus.cc:2454] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0480e933b2b940779e68356e9f76d962 in term 1.
I20260810 00:54:32.411334 30960 leader_election.cc:302] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0480e933b2b940779e68356e9f76d962, 0a60731ddf814efda7c9022f2cca97ac; no voters:
I20260810 00:54:32.411571 31183 raft_consensus.cc:2781] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:54:32.411990 31183 raft_consensus.cc:686] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [term 1 LEADER]: Becoming Leader. State: Replica: 0480e933b2b940779e68356e9f76d962, State: Running, Role: LEADER
I20260810 00:54:32.412417 31183 consensus_queue.cc:237] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0480e933b2b940779e68356e9f76d962" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 42325 } } peers { permanent_uuid: "0a60731ddf814efda7c9022f2cca97ac" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42267 } } peers { permanent_uuid: "843b48a518084bd4bfeada56d9f0a671" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36479 } }
I20260810 00:54:32.416836 30434 catalog_manager.cc:5495] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0480e933b2b940779e68356e9f76d962 (127.10.97.67). New cstate: current_term: 1 leader_uuid: "0480e933b2b940779e68356e9f76d962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0480e933b2b940779e68356e9f76d962" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 42325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a60731ddf814efda7c9022f2cca97ac" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843b48a518084bd4bfeada56d9f0a671" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36479 } health_report { overall_health: UNKNOWN } } }
I20260810 00:54:32.463861 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:32.458457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42960 (local address 127.10.97.67:42325)
0810 00:54:32.458665 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:32.458671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:32.458872 (+ 201us) server_negotiation.cc:407] Connection header received
0810 00:54:32.459072 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:32.459076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:32.459148 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:32.459281 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:54:32.459658 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.460421 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.461799 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.462185 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.462302 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:32.462795 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:54:32.463280 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:54:32.463410 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:32.463512 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 00:54:32.463590 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 00:54:32.471304 30856 raft_consensus.cc:1261] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac [term 1 FOLLOWER]: Refusing update from remote peer 0480e933b2b940779e68356e9f76d962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:54:32.471304 30652 raft_consensus.cc:1261] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671 [term 1 FOLLOWER]: Refusing update from remote peer 0480e933b2b940779e68356e9f76d962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:54:32.472236 31198 consensus_queue.cc:1035] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843b48a518084bd4bfeada56d9f0a671" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:54:32.472594 31183 consensus_queue.cc:1035] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a60731ddf814efda7c9022f2cca97ac" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 42267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:54:32.484319 31206 mvcc.cc:205] Tried to move back new op lower bound from 7316780124033359872 to 7316780123804950528. Current Snapshot: MvccSnapshot[applied={T|T < 7316780124033359872}]
I20260810 00:54:32.485826 31204 mvcc.cc:205] Tried to move back new op lower bound from 7316780124033359872 to 7316780123804950528. Current Snapshot: MvccSnapshot[applied={T|T < 7316780124033359872}]
I20260810 00:54:32.496001 30431 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:54:32.496229 30431 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:54:32.499315 30431 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 118001e1cd7943468314202ab030828a: Sending DeleteTablet for 3 replicas of tablet 3b2ec717e5374c0cabb1fb657e63efdf
I20260810 00:54:32.500355 31031 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b2ec717e5374c0cabb1fb657e63efdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:54:32 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:42946
I20260810 00:54:32.500553 30828 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b2ec717e5374c0cabb1fb657e63efdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:54:32 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:38480
I20260810 00:54:32.500694 30623 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b2ec717e5374c0cabb1fb657e63efdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:54:32 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:43340
I20260810 00:54:32.501549 31224 tablet_replica.cc:330] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac: stopping tablet replica
I20260810 00:54:32.501940 31224 raft_consensus.cc:2229] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:54:32.502413 31224 raft_consensus.cc:2258] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:54:32.503134 31221 tablet_replica.cc:330] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962: stopping tablet replica
I20260810 00:54:32.503353 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 30484
I20260810 00:54:32.503489 31221 raft_consensus.cc:2229] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:54:32.504143 31221 raft_consensus.cc:2258] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:54:32.506228 31224 ts_tablet_manager.cc:1905] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:54:32.506289 31221 ts_tablet_manager.cc:1905] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:54:32.513501 31221 ts_tablet_manager.cc:1918] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:54:32.513736 31221 log.cc:1196] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/3b2ec717e5374c0cabb1fb657e63efdf
I20260810 00:54:32.513906 31224 ts_tablet_manager.cc:1918] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:54:32.514014 31228 tablet_replica.cc:330] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671: stopping tablet replica
I20260810 00:54:32.514178 31224 log.cc:1196] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/3b2ec717e5374c0cabb1fb657e63efdf
I20260810 00:54:32.514274 31221 ts_tablet_manager.cc:1939] T 3b2ec717e5374c0cabb1fb657e63efdf P 0480e933b2b940779e68356e9f76d962: Deleting consensus metadata
I20260810 00:54:32.514417 31228 raft_consensus.cc:2229] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:54:32.514884 31228 raft_consensus.cc:2258] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:54:32.516108 30409 catalog_manager.cc:4841] TS 0480e933b2b940779e68356e9f76d962 (127.10.97.67:42325): tablet 3b2ec717e5374c0cabb1fb657e63efdf (table test-table [id=d9c7cedaccf042398964b05693b47508]) successfully deleted
I20260810 00:54:32.516352 31224 ts_tablet_manager.cc:1939] T 3b2ec717e5374c0cabb1fb657e63efdf P 0a60731ddf814efda7c9022f2cca97ac: Deleting consensus metadata
I20260810 00:54:32.517027 31228 ts_tablet_manager.cc:1905] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:54:32.518013 30406 catalog_manager.cc:4841] TS 0a60731ddf814efda7c9022f2cca97ac (127.10.97.66:42267): tablet 3b2ec717e5374c0cabb1fb657e63efdf (table test-table [id=d9c7cedaccf042398964b05693b47508]) successfully deleted
I20260810 00:54:32.522377 31228 ts_tablet_manager.cc:1918] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:54:32.522586 31228 log.cc:1196] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/3b2ec717e5374c0cabb1fb657e63efdf
I20260810 00:54:32.523097 31228 ts_tablet_manager.cc:1939] T 3b2ec717e5374c0cabb1fb657e63efdf P 843b48a518084bd4bfeada56d9f0a671: Deleting consensus metadata
I20260810 00:54:32.524786 30406 catalog_manager.cc:4841] TS 843b48a518084bd4bfeada56d9f0a671 (127.10.97.65:36479): tablet 3b2ec717e5374c0cabb1fb657e63efdf (table test-table [id=d9c7cedaccf042398964b05693b47508]) successfully deleted
I20260810 00:54:32.528321 31172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:32.504479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43352 (local address 127.10.97.65:36479)
0810 00:54:32.504645 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:32.504651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:32.512310 (+ 7659us) server_negotiation.cc:407] Connection header received
0810 00:54:32.512497 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:32.512504 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:32.512591 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:32.512674 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:32.514870 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.516224 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.518060 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.518690 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.518818 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:32.522152 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:32.522165 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:32.522182 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:32.522230 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:32.525656 (+ 3426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:32.526448 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:32.526455 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:32.526457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:32.526591 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:32.526953 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:32.526956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:32.526958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:32.527295 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:32.527530 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:32.527890 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 00:54:32.528041 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 00:54:32.528856 30705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:32.675930 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 30484
I20260810 00:54:32.698750 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 30718
I20260810 00:54:32.713653 31169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:32.699547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38486 (local address 127.10.97.66:42267)
0810 00:54:32.699723 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:32.699727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:32.699750 (+ 23us) server_negotiation.cc:407] Connection header received
0810 00:54:32.699969 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:32.699973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:32.700040 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:32.700118 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:32.701831 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.702515 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.704510 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.704907 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.704997 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:32.708187 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:32.708201 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:32.708218 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:32.708250 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:32.710886 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:32.711694 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:32.711700 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:32.711703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:32.711952 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:32.712316 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:32.712319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:32.712321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:32.712685 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:32.712896 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:32.713220 (+ 324us) server_negotiation.cc:299] Negotiation successful
0810 00:54:32.713370 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 00:54:32.714304 30918 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:32.801689 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 30718
I20260810 00:54:32.832118 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 30935
I20260810 00:54:32.849824 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:32.833148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42966 (local address 127.10.97.67:42325)
0810 00:54:32.833353 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:32.833359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:32.833385 (+ 26us) server_negotiation.cc:407] Connection header received
0810 00:54:32.833460 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:32.833466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:32.833575 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:32.833691 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:32.835396 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.836655 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.838487 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:32.839153 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:32.839294 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:32.842521 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:32.842535 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:32.842553 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:32.842607 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:32.846779 (+ 4172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:32.847537 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:32.847547 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:32.847550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:32.847750 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:32.848186 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:32.848191 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:32.848195 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:32.848618 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:32.848947 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:32.849195 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 00:54:32.849409 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 00:54:32.850610 31124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:32.982990 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 30935
I20260810 00:54:33.006925 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 30379
I20260810 00:54:33.008070 30475 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:33.114830 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 30379
I20260810 00:54:33.143380 10629 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 00:54:33.488042 10629 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 00:54:33.488256 10629 mini_ranger.cc:66] Stopping Ranger...
I20260810 00:54:33.831737 10629 mini_ranger.cc:68] Stopped Ranger
2026-08-10 00:54:33.832 UTC [28908] LOG: received smart shutdown request
2026-08-10 00:54:33.834 UTC [28908] LOG: background worker "logical replication launcher" (PID 28932) exited with exit code 1
2026-08-10 00:54:33.834 UTC [28920] LOG: shutting down
2026-08-10 00:54:33.953 UTC [28908] LOG: database system is shut down
2026-08-10T00:54:33Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49464 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:54:34 dist-test-slave-jlzn krb5kdc[31244](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:54:34 dist-test-slave-jlzn krb5kdc[31244](info): set up 2 sockets
Aug 10 00:54:34 dist-test-slave-jlzn krb5kdc[31244](info): commencing operation
krb5kdc: starting...
W20260810 00:54:35.265954 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.005s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:54:35 dist-test-slave-jlzn krb5kdc[31244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:54:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:54:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:54:35.486460 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:35089
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:35089,127.10.97.125:38149,127.10.97.124:36497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:54:35.798691 31261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:35.798981 31261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:35.799111 31261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:35.806429 31261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:54:35.806556 31261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:35.806638 31261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:35.806717 31261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:54:35.806779 31261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:54:35.816385 31261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:35089,127.10.97.125:38149,127.10.97.124:36497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:35089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:35.817615 31261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:35.818857 31261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:35.829890 31268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:35.829922 31271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:35.830155 31267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:35.830755 31261 server_base.cc:1034] running on GCE node
I20260810 00:54:35.831480 31261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:35.832818 31261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:35.834024 31261 hybrid_clock.cc:648] HybridClock initialized: now 1786323275833978 us; error 28 us; skew 500 ppm
I20260810 00:54:35.834337 31261 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:35 dist-test-slave-jlzn krb5kdc[31244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323275, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:35.841028 31261 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:54:35.842396 31261 webserver.cc:460] Webserver started at http://127.10.97.126:36265/ using document root <none> and password file <none>
I20260810 00:54:35.843046 31261 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:35.843174 31261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:35.843457 31261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:54:35.846383 31261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "e424e98e56a945d3bc07e995d514b3fd"
format_stamp: "Formatted at 2026-08-10 00:54:35 on dist-test-slave-jlzn"
server_key: "2e020d05d952da0106bca2527f217c04"
server_key_iv: "0a623dc54701b185e5af5efe02f05c00"
server_key_version: "encryptionkey@0"
I20260810 00:54:35.847173 31261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "e424e98e56a945d3bc07e995d514b3fd"
format_stamp: "Formatted at 2026-08-10 00:54:35 on dist-test-slave-jlzn"
server_key: "2e020d05d952da0106bca2527f217c04"
server_key_iv: "0a623dc54701b185e5af5efe02f05c00"
server_key_version: "encryptionkey@0"
I20260810 00:54:35.852129 31261 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 00:54:35.855360 31283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:35.856626 31261 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:54:35.856782 31261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "e424e98e56a945d3bc07e995d514b3fd"
format_stamp: "Formatted at 2026-08-10 00:54:35 on dist-test-slave-jlzn"
server_key: "2e020d05d952da0106bca2527f217c04"
server_key_iv: "0a623dc54701b185e5af5efe02f05c00"
server_key_version: "encryptionkey@0"
I20260810 00:54:35.856956 31261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:35.879377 31261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:35.883369 31261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:35.883648 31261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:35.901227 31261 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:35089
I20260810 00:54:35.901288 31363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:35089 every 8 connection(s)
I20260810 00:54:35.902460 31261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:54:35.904078 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 31261
I20260810 00:54:35.904325 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:54:35.905166 10629 external_mini_cluster.cc:1428] Setting key 0428272ff378f02b2c968878550b562e
I20260810 00:54:35.906699 31365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:54:35.914927 31365 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:54:35 dist-test-slave-jlzn krb5kdc[31244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323275, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.97.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.125@KRBTEST.COM" created.
I20260810 00:54:35.935413 31373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:35.918096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33016 (local address 127.10.97.126:35089)
0810 00:54:35.919653 (+ 1557us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:35.919660 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:35.919710 (+ 50us) server_negotiation.cc:407] Connection header received
0810 00:54:35.921151 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:35.921156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:35.921348 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:35.921519 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:35.922771 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:35.923976 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:35.925285 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:35.925804 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:35.925981 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:35.929135 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:35.929143 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:35.929153 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:35.929184 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:35.931733 (+ 2549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:35.932368 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:35.932374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:35.932376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:35.932543 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:35.932975 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:35.932979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:35.932982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:35.933434 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:35.933591 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:35.934214 (+ 623us) server_negotiation.cc:299] Negotiation successful
0810 00:54:35.934442 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1336,"spinlock_wait_cycles":15232,"thread_start_us":1146,"threads_started":1}
I20260810 00:54:35.935456 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:35.918094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:35089 (local address 127.0.0.1:33016)
0810 00:54:35.918879 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:35.918924 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:35.920821 (+ 1897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:35.921548 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:35.921554 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:35.922217 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:35.922591 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:35.922609 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:35.924110 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:35.924114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:35.925158 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:35.925170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:35.926024 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:35.926031 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:35.926251 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:35.926913 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:35.926939 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:35.928937 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:35.931860 (+ 2923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:35.931871 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:35.931881 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:35.932255 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:35.932640 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:35.932645 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:35.932649 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:35.932836 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:35.933581 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:35.933589 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:35.933838 (+ 249us) client_negotiation.cc:769] Sending connection context
0810 00:54:35.934080 (+ 242us) client_negotiation.cc:240] Negotiation successful
0810 00:54:35.934592 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":133,"thread_start_us":342,"threads_started":1}
I20260810 00:54:35.937903 31365 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:54:35.938757 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:35.938434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:38149 (local address 127.0.0.1:54954)
0810 00:54:35.938558 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:35.938664 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:38149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 00:54:35.938978 31284 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.97.125:38149: connect: Connection refused (error 111)
I20260810 00:54:35.940434 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:35.940115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:38149 (local address 127.0.0.1:54968)
0810 00:54:35.940245 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:35.940309 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:38149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 00:54:35.940614 31365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.125:38149: Network error: Client connection negotiation failed: client connection to 127.10.97.125:38149: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.125.keytab.
Entry for principal kudu/127.10.97.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.125.keytab.
I20260810 00:54:35.958424 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.125:38149
--webserver_interface=127.10.97.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.125.keytab
--principal=kudu/127.10.97.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:35089,127.10.97.125:38149,127.10.97.124:36497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:54:35.989930 31365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } attempt: 1
I20260810 00:54:35.990993 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:35.990574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:38149 (local address 127.0.0.1:54974)
0810 00:54:35.990753 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:35.990871 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:38149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 00:54:35.991946 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:35.991592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:38149 (local address 127.0.0.1:54976)
0810 00:54:35.991722 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:35.991787 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:38149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 00:54:35.992157 31365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.125:38149: Network error: Client connection negotiation failed: client connection to 127.10.97.125:38149: connect: Connection refused (error 111)
I20260810 00:54:36.060380 31365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } attempt: 2
I20260810 00:54:36.061600 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.061100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:38149 (local address 127.0.0.1:54978)
0810 00:54:36.061323 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.061440 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:38149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 00:54:36.062638 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.062283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:38149 (local address 127.0.0.1:54982)
0810 00:54:36.062438 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.062511 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:38149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 00:54:36.062814 31365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.125:38149: Network error: Client connection negotiation failed: client connection to 127.10.97.125:38149: connect: Connection refused (error 111)
I20260810 00:54:36.154055 31365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } attempt: 3
I20260810 00:54:36.155138 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.154701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:38149 (local address 127.0.0.1:54998)
0810 00:54:36.154890 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.154996 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:38149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 00:54:36.156544 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.156107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:38149 (local address 127.0.0.1:55008)
0810 00:54:36.156280 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.156375 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:38149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 00:54:36.156870 31365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.125:38149: Network error: Client connection negotiation failed: client connection to 127.10.97.125:38149: connect: Connection refused (error 111)
W20260810 00:54:36.269758 31376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:36.270015 31376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:36.270107 31376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:36.276870 31376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:54:36.276961 31376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:36.277014 31376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:36.277069 31376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:54:36.277112 31376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:54:36.286168 31376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:35089,127.10.97.125:38149,127.10.97.124:36497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.125.keytab
--principal=kudu/127.10.97.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.125:38149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.97.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:36.287272 31376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:36.288442 31376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:36.298898 31383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:36.298964 31385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:36.298996 31389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:36.300072 31376 server_base.cc:1034] running on GCE node
I20260810 00:54:36.300114 31365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } attempt: 4
I20260810 00:54:36.300930 31376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:36.301250 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.300740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:38149 (local address 127.0.0.1:55016)
0810 00:54:36.301015 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.301110 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:38149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260810 00:54:36.302163 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.301813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:38149 (local address 127.0.0.1:55032)
0810 00:54:36.301956 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.302043 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:38149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 00:54:36.302368 31376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
W20260810 00:54:36.302377 31365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.125:38149: Network error: Client connection negotiation failed: client connection to 127.10.97.125:38149: connect: Connection refused (error 111)
I20260810 00:54:36.303580 31376 hybrid_clock.cc:648] HybridClock initialized: now 1786323276303532 us; error 41 us; skew 500 ppm
I20260810 00:54:36.303916 31376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:36 dist-test-slave-jlzn krb5kdc[31244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323276, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:36.306878 31376 init.cc:377] Logged in from keytab as kudu/127.10.97.125@KRBTEST.COM (short username kudu)
I20260810 00:54:36.308184 31376 webserver.cc:460] Webserver started at http://127.10.97.125:41061/ using document root <none> and password file <none>
I20260810 00:54:36.308792 31376 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:36.308902 31376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:36.309165 31376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:54:36.311976 31376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-1/data/instance:
uuid: "a5d2ef87358e4675a5ba136104201c3a"
format_stamp: "Formatted at 2026-08-10 00:54:36 on dist-test-slave-jlzn"
server_key: "3783176fc6a59d623220dd94908d8e7b"
server_key_iv: "00e3e1a11869c0494ca40ce1af4356d1"
server_key_version: "encryptionkey@0"
I20260810 00:54:36.312726 31376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-1/wal/instance:
uuid: "a5d2ef87358e4675a5ba136104201c3a"
format_stamp: "Formatted at 2026-08-10 00:54:36 on dist-test-slave-jlzn"
server_key: "3783176fc6a59d623220dd94908d8e7b"
server_key_iv: "00e3e1a11869c0494ca40ce1af4356d1"
server_key_version: "encryptionkey@0"
I20260810 00:54:36.317438 31376 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:54:36.320487 31401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:36.321648 31376 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:54:36.321801 31376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-1/wal
uuid: "a5d2ef87358e4675a5ba136104201c3a"
format_stamp: "Formatted at 2026-08-10 00:54:36 on dist-test-slave-jlzn"
server_key: "3783176fc6a59d623220dd94908d8e7b"
server_key_iv: "00e3e1a11869c0494ca40ce1af4356d1"
server_key_version: "encryptionkey@0"
I20260810 00:54:36.321965 31376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:36.355876 31376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:36.359881 31376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:36.360224 31376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:36.376574 31376 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.125:38149
I20260810 00:54:36.376634 31471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.125:38149 every 8 connection(s)
I20260810 00:54:36.377785 31376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-1/data/info.pb
I20260810 00:54:36.381110 31472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:54:36.387210 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 31376
I20260810 00:54:36.387379 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-1/wal/instance
I20260810 00:54:36.387847 10629 external_mini_cluster.cc:1428] Setting key 1da93d45ec8fb748180af7bebaa7a451
I20260810 00:54:36.387135 31472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:54:36 dist-test-slave-jlzn krb5kdc[31244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323276, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.125@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:54:36.406553 31373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.389536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33022 (local address 127.10.97.126:35089)
0810 00:54:36.389688 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:36.389693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:36.390881 (+ 1188us) server_negotiation.cc:407] Connection header received
0810 00:54:36.392702 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:36.392708 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:36.392781 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:36.392868 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:36.394251 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.394922 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.396477 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.397028 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.397171 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:36.400531 (+ 3360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:36.400537 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:36.400541 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:36.400579 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:36.403644 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.404387 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.404393 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.404396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.404538 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.404897 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.404903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.404906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.405280 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:36.405424 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:36.405994 (+ 570us) server_negotiation.cc:299] Negotiation successful
0810 00:54:36.406246 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 00:54:36.407131 31474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.390045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:35089 (local address 127.0.0.1:33022)
0810 00:54:36.390733 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.390768 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:36.392440 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:36.393102 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:36.393109 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:36.393573 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:36.394077 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:36.394095 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.395073 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.395077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:36.396320 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:36.396337 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.397226 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.397231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:36.397442 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:36.398074 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:36.398103 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:36.400316 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:36.403766 (+ 3450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:36.403772 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:36.403781 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:36.404250 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:36.404632 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:36.404635 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:36.404637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:36.404788 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:36.405387 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:36.405392 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:36.405653 (+ 261us) client_negotiation.cc:769] Sending connection context
0810 00:54:36.405850 (+ 197us) client_negotiation.cc:240] Negotiation successful
0810 00:54:36.406182 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":292,"threads_started":1}
I20260810 00:54:36.408767 31472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.97.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.124@KRBTEST.COM" created.
Aug 10 00:54:36 dist-test-slave-jlzn krb5kdc[31244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323276, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.125@KRBTEST.COM for kudu/127.10.97.125@KRBTEST.COM
I20260810 00:54:36.422233 31474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.409322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:38149 (local address 127.0.0.1:55034)
0810 00:54:36.409472 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.409492 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:36.409574 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:36.410617 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:36.410621 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:36.411012 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:36.411131 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:36.411139 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.412466 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.412470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:36.413258 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:36.413268 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.414066 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.414072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:36.414264 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:36.414726 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:36.414741 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:36.417032 (+ 2291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:36.419824 (+ 2792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:36.419829 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:36.419832 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:36.420281 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:36.420609 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:36.420613 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:36.420616 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:36.420803 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:36.421341 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:36.421345 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:36.421485 (+ 140us) client_negotiation.cc:769] Sending connection context
0810 00:54:36.421619 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 00:54:36.421886 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 00:54:36.422282 31478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.409676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55034 (local address 127.10.97.125:38149)
0810 00:54:36.410275 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:36.410281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:36.410304 (+ 23us) server_negotiation.cc:407] Connection header received
0810 00:54:36.410341 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:36.410346 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:36.410488 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:36.410601 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:36.411277 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.412345 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.413380 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.413942 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.414046 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:36.417226 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:36.417236 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:36.417247 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:36.417275 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:36.419679 (+ 2404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.420392 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.420397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.420399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.420514 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.420904 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.420907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.420909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.421222 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:36.421369 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:36.421723 (+ 354us) server_negotiation.cc:299] Negotiation successful
0810 00:54:36.421988 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":471,"thread_start_us":335,"threads_started":1}
I20260810 00:54:36.424463 31472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:54:36.425534 31474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.425051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58220)
0810 00:54:36.425191 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.425356 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 00:54:36.425751 31406 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
I20260810 00:54:36.427245 31474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.426885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58226)
0810 00:54:36.427055 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.427137 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 00:54:36.427453 31472 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:36497: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.124.keytab.
Entry for principal kudu/127.10.97.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.124.keytab.
I20260810 00:54:36.442476 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.124:36497
--webserver_interface=127.10.97.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.124.keytab
--principal=kudu/127.10.97.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:35089,127.10.97.125:38149,127.10.97.124:36497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:54:36.476766 31472 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } attempt: 1
I20260810 00:54:36.477984 31474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.477452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58234)
0810 00:54:36.477675 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.477799 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 00:54:36.478941 31474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.478565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58248)
0810 00:54:36.478733 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.478813 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 00:54:36.479136 31472 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:36497: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
I20260810 00:54:36.547365 31472 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } attempt: 2
I20260810 00:54:36.548626 31474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.548073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58258)
0810 00:54:36.548287 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.548431 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 00:54:36.549652 31474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.549261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58272)
0810 00:54:36.549408 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.549494 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 00:54:36.549870 31472 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:36497: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
I20260810 00:54:36.601606 31365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } attempt: 5
Aug 10 00:54:36 dist-test-slave-jlzn krb5kdc[31244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323275, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for kudu/127.10.97.125@KRBTEST.COM
I20260810 00:54:36.615247 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.602323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:38149 (local address 127.0.0.1:55048)
0810 00:54:36.602557 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.602576 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:36.602689 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:36.603026 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:36.603029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:36.603405 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:36.603571 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:36.603581 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.605005 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.605009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:36.605869 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:36.605881 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.606863 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.606867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:36.606999 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:36.607585 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:36.607603 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:36.610049 (+ 2446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:36.612908 (+ 2859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:36.612913 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:36.612917 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:36.613410 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:36.613725 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:36.613730 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:36.613734 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:36.613901 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:36.614551 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:36.614555 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:36.614706 (+ 151us) client_negotiation.cc:769] Sending connection context
0810 00:54:36.614821 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 00:54:36.615008 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 00:54:36.615502 31478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.602473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55048 (local address 127.10.97.125:38149)
0810 00:54:36.602669 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:36.602676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:36.602699 (+ 23us) server_negotiation.cc:407] Connection header received
0810 00:54:36.602789 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:36.602794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:36.602897 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:36.603006 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:36.603752 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.604875 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.606023 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.606730 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.606865 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:36.610227 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:36.610231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:36.610234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:36.610266 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:36.612774 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.613507 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.613510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.613512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.613620 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.614039 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.614044 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.614048 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.614425 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:36.614587 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:36.614968 (+ 381us) server_negotiation.cc:299] Negotiation successful
0810 00:54:36.615200 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 00:54:36.616439 31365 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:54:36.617378 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.617018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58274)
0810 00:54:36.617175 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.617249 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 00:54:36.618280 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.617990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58280)
0810 00:54:36.618098 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.618185 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 00:54:36.618499 31365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:36497: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
I20260810 00:54:36.641096 31472 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } attempt: 3
I20260810 00:54:36.642195 31474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.641673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58284)
0810 00:54:36.641882 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.641996 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 00:54:36.643321 31474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.642830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58300)
0810 00:54:36.643072 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.643173 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260810 00:54:36.643543 31472 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:36497: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
I20260810 00:54:36.669697 31365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } attempt: 1
I20260810 00:54:36.670661 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.670271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58306)
0810 00:54:36.670425 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.670521 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 00:54:36.671515 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.671195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58322)
0810 00:54:36.671331 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.671398 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 00:54:36.671710 31365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:36497: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
I20260810 00:54:36.739916 31365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } attempt: 2
I20260810 00:54:36.740971 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.740497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58336)
0810 00:54:36.740720 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.740806 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 00:54:36.741919 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.741576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58340)
0810 00:54:36.741732 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.741809 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 00:54:36.742107 31365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:36497: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
W20260810 00:54:36.757645 31484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:36.757906 31484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:36.758040 31484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:36.765177 31484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:54:36.765278 31484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:36.765357 31484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:36.765434 31484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:54:36.765503 31484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:54:36.774775 31484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:35089,127.10.97.125:38149,127.10.97.124:36497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.124.keytab
--principal=kudu/127.10.97.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.124:36497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.97.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:36.776012 31484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:36.777238 31484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 00:54:36.786741 31472 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } attempt: 4
I20260810 00:54:36.787948 31474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.787378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58342)
0810 00:54:36.787613 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.787719 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 00:54:36.788224 31494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:36.788534 31491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:36.788812 31497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:36.789168 31474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.788679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58350)
0810 00:54:36.788943 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.789037 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
W20260810 00:54:36.789433 31472 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:36497: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
I20260810 00:54:36.789769 31484 server_base.cc:1034] running on GCE node
I20260810 00:54:36.790468 31484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:36.792210 31484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:36.793409 31484 hybrid_clock.cc:648] HybridClock initialized: now 1786323276793368 us; error 29 us; skew 500 ppm
I20260810 00:54:36.793732 31484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:36 dist-test-slave-jlzn krb5kdc[31244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323276, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:36.796676 31484 init.cc:377] Logged in from keytab as kudu/127.10.97.124@KRBTEST.COM (short username kudu)
I20260810 00:54:36.798135 31484 webserver.cc:460] Webserver started at http://127.10.97.124:43393/ using document root <none> and password file <none>
I20260810 00:54:36.798799 31484 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:36.798910 31484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:36.799196 31484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:54:36.802119 31484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-2/data/instance:
uuid: "50ceb7989897418789fbdbe635e0ef8c"
format_stamp: "Formatted at 2026-08-10 00:54:36 on dist-test-slave-jlzn"
server_key: "b7c58eed8eb0b06df445b3326565941e"
server_key_iv: "eeb0a5485f7eafa0db5b6c27006ba732"
server_key_version: "encryptionkey@0"
I20260810 00:54:36.802891 31484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-2/wal/instance:
uuid: "50ceb7989897418789fbdbe635e0ef8c"
format_stamp: "Formatted at 2026-08-10 00:54:36 on dist-test-slave-jlzn"
server_key: "b7c58eed8eb0b06df445b3326565941e"
server_key_iv: "eeb0a5485f7eafa0db5b6c27006ba732"
server_key_version: "encryptionkey@0"
I20260810 00:54:36.807739 31484 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260810 00:54:36.810904 31506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:36.812245 31484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:54:36.812403 31484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-2/wal
uuid: "50ceb7989897418789fbdbe635e0ef8c"
format_stamp: "Formatted at 2026-08-10 00:54:36 on dist-test-slave-jlzn"
server_key: "b7c58eed8eb0b06df445b3326565941e"
server_key_iv: "eeb0a5485f7eafa0db5b6c27006ba732"
server_key_version: "encryptionkey@0"
I20260810 00:54:36.812582 31484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:36.831557 31484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:36.835083 31484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:36.835335 31484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:36.848289 31365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } attempt: 3
I20260810 00:54:36.849443 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.848870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58362)
0810 00:54:36.849123 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.849240 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 00:54:36.850996 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.850592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58374)
0810 00:54:36.850812 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.850886 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260810 00:54:36.851284 31365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:36497: Network error: Client connection negotiation failed: client connection to 127.10.97.124:36497: connect: Connection refused (error 111)
I20260810 00:54:36.854087 31484 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.124:36497
I20260810 00:54:36.854146 31584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.124:36497 every 8 connection(s)
I20260810 00:54:36.855283 31484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-2/data/info.pb
I20260810 00:54:36.858752 31587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:54:36.862619 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 31484
I20260810 00:54:36.862807 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786323077328248-10629-0/minicluster-data/master-2/wal/instance
I20260810 00:54:36.863238 10629 external_mini_cluster.cc:1428] Setting key 9defa4c7a49a9a47de6f99184f4fbe34
I20260810 00:54:36.865896 31587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:54:36 dist-test-slave-jlzn krb5kdc[31244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323276, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.124@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
Aug 10 00:54:36 dist-test-slave-jlzn krb5kdc[31244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.124@KRBTEST.COM
I20260810 00:54:36.885720 31373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.869489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33030 (local address 127.10.97.126:35089)
0810 00:54:36.869660 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:36.869666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:36.870385 (+ 719us) server_negotiation.cc:407] Connection header received
0810 00:54:36.872400 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:36.872430 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:36.872538 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:36.872684 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:36.874085 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.874769 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.876256 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.876693 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.876805 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:36.880135 (+ 3330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:36.880141 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:36.880144 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:36.880175 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:36.882807 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.883560 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.883565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.883568 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.883691 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.884104 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.884108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.884110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.884358 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:36.884503 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:36.885161 (+ 658us) server_negotiation.cc:299] Negotiation successful
0810 00:54:36.885382 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 00:54:36.886643 31592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.865850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58386 (local address 127.10.97.124:36497)
0810 00:54:36.869069 (+ 3219us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:36.869083 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:36.869130 (+ 47us) server_negotiation.cc:407] Connection header received
0810 00:54:36.872115 (+ 2985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:36.872143 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:36.872422 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:36.872594 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:36.874085 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.875226 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.876577 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.877092 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.877218 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:36.881108 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:36.881116 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:36.881126 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:36.881159 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:36.883752 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.884430 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.884436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.884438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.884573 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.884938 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.884941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.884943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.885271 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:36.885561 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:36.885880 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 00:54:36.886103 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2981,"thread_start_us":2735,"threads_started":1}
I20260810 00:54:36.886682 31593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.869459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:35089 (local address 127.0.0.1:33030)
0810 00:54:36.870232 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.870282 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:36.872121 (+ 1839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:36.872917 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:36.872924 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:36.873541 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:36.873924 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:36.873940 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.874929 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.874934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:36.876114 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:36.876126 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.876921 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.876929 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:36.877108 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:36.877737 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:36.877765 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:36.879953 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:36.882930 (+ 2977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:36.882941 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:36.882952 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:36.883440 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:36.883796 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:36.883819 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:36.883823 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:36.883992 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:36.884485 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:36.884493 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:36.884792 (+ 299us) client_negotiation.cc:769] Sending connection context
0810 00:54:36.885007 (+ 215us) client_negotiation.cc:240] Negotiation successful
0810 00:54:36.885439 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":399,"threads_started":1}
I20260810 00:54:36.888274 31587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:54:36 dist-test-slave-jlzn krb5kdc[31244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323276, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.124@KRBTEST.COM for kudu/127.10.97.125@KRBTEST.COM
Aug 10 00:54:36 dist-test-slave-jlzn krb5kdc[31244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:54:36.901006 31593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.888850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:38149 (local address 127.0.0.1:55058)
0810 00:54:36.889025 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.889042 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:36.889137 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:36.889636 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:36.889641 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:36.889959 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:36.890148 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:36.890158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.891079 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.891085 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:36.892031 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:36.892045 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.892712 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.892718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:36.892922 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:36.893417 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:36.893432 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:36.895418 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:36.898668 (+ 3250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:36.898674 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:36.898678 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:36.899054 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:36.899433 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:36.899435 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:36.899438 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:36.899597 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:36.900237 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:36.900241 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:36.900356 (+ 115us) client_negotiation.cc:769] Sending connection context
0810 00:54:36.900457 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 00:54:36.900705 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 00:54:36.901443 31478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.889150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55058 (local address 127.10.97.125:38149)
0810 00:54:36.889319 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:36.889325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:36.889344 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:54:36.889401 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:36.889406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:36.889481 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:36.889590 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:36.890298 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.890947 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.892163 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.892578 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.892668 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:36.895605 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:36.895610 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:36.895613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:36.895645 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:36.898529 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.899163 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.899168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.899171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.899336 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.899743 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.899748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.899751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.900124 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:36.900271 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:36.900597 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 00:54:36.900804 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 00:54:36.902285 31587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:54:36.904345 31373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.889568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33038 (local address 127.10.97.126:35089)
0810 00:54:36.889739 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:36.889744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:36.889758 (+ 14us) server_negotiation.cc:407] Connection header received
0810 00:54:36.889793 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:36.889796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:36.889842 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:36.889911 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:36.891093 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.891827 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.893307 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.893752 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.893843 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:36.898024 (+ 4181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:36.898030 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:36.898035 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:36.898073 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:36.901231 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.901995 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.902000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.902004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.902167 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.902531 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.902536 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.902538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.902852 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:36.903166 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:36.903940 (+ 774us) server_negotiation.cc:299] Negotiation successful
0810 00:54:36.904095 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 00:54:36 dist-test-slave-jlzn krb5kdc[31244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323276, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.124@KRBTEST.COM for kudu/127.10.97.124@KRBTEST.COM
Aug 10 00:54:36 dist-test-slave-jlzn krb5kdc[31244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.125@KRBTEST.COM
I20260810 00:54:36.915784 31593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.902915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58400)
0810 00:54:36.903205 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:36.903219 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:36.903296 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:36.903553 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:36.903557 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:36.904005 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:36.904163 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:36.904173 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.905165 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.905170 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:36.906267 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:36.906281 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.907113 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.907119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:36.907312 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:36.907890 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:36.907910 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:36.910345 (+ 2435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:36.913418 (+ 3073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:36.913421 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:36.913424 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:36.913872 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:36.914217 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:36.914222 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:36.914225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:36.914385 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:36.914880 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:36.914885 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:36.915014 (+ 129us) client_negotiation.cc:769] Sending connection context
0810 00:54:36.915140 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 00:54:36.915405 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201}
I20260810 00:54:36.916031 31592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.903081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58400 (local address 127.10.97.124:36497)
0810 00:54:36.903305 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:36.903310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:36.903326 (+ 16us) server_negotiation.cc:407] Connection header received
0810 00:54:36.903370 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:36.903374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:36.903437 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:36.903526 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:36.904306 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.905055 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.906371 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.906950 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.907087 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:36.910527 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:36.910532 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:36.910537 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:36.910569 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:36.913294 (+ 2725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.913983 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.913987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.913989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.914096 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.914508 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.914512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.914514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.914772 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:36.914895 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:36.915232 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 00:54:36.915538 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":125,"server-negotiator.queue_time_us":118}
I20260810 00:54:36.919734 31478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.905898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55072 (local address 127.10.97.125:38149)
0810 00:54:36.906041 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:36.906046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:36.906134 (+ 88us) server_negotiation.cc:407] Connection header received
0810 00:54:36.906300 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:36.906305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:36.906369 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:36.906456 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:36.907951 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.908624 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.909811 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.910211 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.910314 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:36.914603 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:36.914607 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:36.914610 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:36.914637 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:36.917406 (+ 2769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.917995 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.918001 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.918004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.918148 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.918480 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.918485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.918489 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.918741 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:36.918976 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:36.919257 (+ 281us) server_negotiation.cc:299] Negotiation successful
0810 00:54:36.919461 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 00:54:36.920554 31587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c: Bootstrap starting.
I20260810 00:54:36.924208 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:54:36.924247 31587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c: Neither blocks nor log segments found. Creating new log.
I20260810 00:54:36.925431 31587 log.cc:824] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c: Log is configured to *not* fsync() on all Append() calls
I20260810 00:54:36.928488 31587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c: No bootstrap required, opened a new log
I20260810 00:54:36.939298 31587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } } peers { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } } peers { permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } }
I20260810 00:54:36.940474 31587 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:54:36.940656 31587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ceb7989897418789fbdbe635e0ef8c, State: Initialized, Role: FOLLOWER
I20260810 00:54:36.941449 31587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } } peers { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } } peers { permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } }
I20260810 00:54:36.948276 31373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.929673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33044 (local address 127.10.97.126:35089)
0810 00:54:36.929832 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:36.929837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:36.930590 (+ 753us) server_negotiation.cc:407] Connection header received
0810 00:54:36.930856 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:36.930861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:36.930934 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:36.931004 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:36.933537 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.934662 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.936872 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.937609 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.937703 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:36.941512 (+ 3809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:36.941519 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:36.941524 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:36.941562 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:36.944880 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.945837 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.945842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.945845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.946006 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.946472 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.946477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.946481 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.946814 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:36.947035 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:36.947662 (+ 627us) server_negotiation.cc:299] Negotiation successful
0810 00:54:36.947918 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 00:54:36.950168 31587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:54:36.951570 31608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } } peers { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } } peers { permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } } }
I20260810 00:54:36.952142 31608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:54:36.954424 31592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.929833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58416 (local address 127.10.97.124:36497)
0810 00:54:36.930020 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:36.930026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:36.936712 (+ 6686us) server_negotiation.cc:407] Connection header received
0810 00:54:36.937098 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:36.937103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:36.937168 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:36.937238 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:36.939631 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.940635 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.942781 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.943539 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.943634 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:36.947211 (+ 3577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:36.947218 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:36.947223 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:36.947263 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:36.950651 (+ 3388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.951579 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.951585 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.951588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.951792 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.952282 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.952287 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.952291 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.952673 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:36.953822 (+ 1149us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:36.953948 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 00:54:36.954147 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 00:54:36.955996 31478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:36.930920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55082 (local address 127.10.97.125:38149)
0810 00:54:36.931062 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:36.931067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:36.940258 (+ 9191us) server_negotiation.cc:407] Connection header received
0810 00:54:36.940413 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:36.940419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:36.940493 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:36.940587 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:36.943009 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.944070 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.946153 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:36.946772 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:36.946866 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:36.949913 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:36.949919 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:36.949924 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:36.949964 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:36.953081 (+ 3117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.953920 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.953925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.953928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.954093 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:36.954574 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:36.954579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:36.954582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:36.954936 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:36.955206 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:36.955558 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 00:54:36.955724 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 00:54:36.962491 31622 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 00:54:36.962625 31622 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 00:54:36.959127 31484 master_runner.cc:186] Error getting master registration for 127.10.97.126:35089: OK, instance_id { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" instance_seqno: 1786323275896394 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 00:54:36.967883 31484 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 00:54:37.028453 31365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } attempt: 4
Aug 10 00:54:37 dist-test-slave-jlzn krb5kdc[31244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323275, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for kudu/127.10.97.124@KRBTEST.COM
I20260810 00:54:37.042490 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:37.028978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58428)
0810 00:54:37.029177 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:37.029198 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:37.029313 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:37.029639 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:37.029644 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:37.030036 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:37.030190 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:37.030199 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:37.031203 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:37.031209 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:37.032236 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:37.032248 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:37.033228 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:37.033234 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:37.033418 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:37.034258 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:37.034283 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:37.036574 (+ 2291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:37.040402 (+ 3828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:37.040406 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:37.040409 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:37.040756 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:37.041136 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:37.041139 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:37.041142 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:37.041260 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:37.041814 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:37.041817 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:37.041939 (+ 122us) client_negotiation.cc:769] Sending connection context
0810 00:54:37.042044 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 00:54:37.042228 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 00:54:37.042748 31592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:37.029024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58428 (local address 127.10.97.124:36497)
0810 00:54:37.029186 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:37.029192 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:37.029258 (+ 66us) server_negotiation.cc:407] Connection header received
0810 00:54:37.029431 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:37.029435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:37.029508 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:37.029589 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:37.030325 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:37.031075 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:37.032364 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:37.032872 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:37.032999 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:37.036755 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:37.036761 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:37.036766 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:37.036809 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:37.040289 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:37.040859 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:37.040865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:37.040869 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:37.041043 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:37.041354 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:37.041359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:37.041362 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:37.041702 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:37.041847 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:37.042162 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 00:54:37.042389 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 00:54:37.047152 31365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd: Bootstrap starting.
I20260810 00:54:37.050611 31365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd: Neither blocks nor log segments found. Creating new log.
I20260810 00:54:37.051692 31365 log.cc:824] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd: Log is configured to *not* fsync() on all Append() calls
I20260810 00:54:37.054315 31365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd: No bootstrap required, opened a new log
I20260810 00:54:37.061146 31365 raft_consensus.cc:348] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } } peers { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } } peers { permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } }
I20260810 00:54:37.061430 31365 raft_consensus.cc:374] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:54:37.061568 31365 raft_consensus.cc:729] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e424e98e56a945d3bc07e995d514b3fd, State: Initialized, Role: FOLLOWER
I20260810 00:54:37.062325 31365 consensus_queue.cc:260] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } } peers { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } } peers { permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } }
I20260810 00:54:37.063352 31628 sys_catalog.cc:455] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } } peers { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } } peers { permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } } }
I20260810 00:54:37.063767 31628 sys_catalog.cc:458] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:54:37.064476 31365 sys_catalog.cc:564] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [sys.catalog]: configured and running, proceeding with master startup.
W20260810 00:54:37.073395 31641 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 00:54:37.073585 31641 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 00:54:37.072810 31261 master_runner.cc:186] Error getting master registration for 127.10.97.125:38149: OK, instance_id { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" instance_seqno: 1786323276371915 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 00:54:37.081522 31261 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 00:54:37.087235 31514 tablet.cc:2369] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:54:37.088639 31472 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } attempt: 5
Aug 10 00:54:37 dist-test-slave-jlzn krb5kdc[31244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323276, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.125@KRBTEST.COM for kudu/127.10.97.124@KRBTEST.COM
I20260810 00:54:37.101668 31474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:37.089111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:36497 (local address 127.0.0.1:58430)
0810 00:54:37.089368 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:37.089386 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:37.089503 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:37.089814 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:37.089819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:37.090220 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:37.090377 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:37.090385 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:37.091372 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:37.091378 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:37.092316 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:37.092328 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:37.092996 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:37.093001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:37.093196 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:37.093721 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:37.093737 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:37.095890 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:37.099393 (+ 3503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:37.099397 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:37.099399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:37.099759 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:37.100129 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:37.100139 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:37.100141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:37.100275 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:37.100771 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:37.100776 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:37.100900 (+ 124us) client_negotiation.cc:769] Sending connection context
0810 00:54:37.101021 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 00:54:37.101293 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260810 00:54:37.101759 31592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:37.089196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58430 (local address 127.10.97.124:36497)
0810 00:54:37.089367 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:37.089373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:37.089455 (+ 82us) server_negotiation.cc:407] Connection header received
0810 00:54:37.089623 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:37.089626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:37.089695 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:37.089770 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:37.090513 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:37.091234 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:37.092448 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:37.092867 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:37.092972 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:37.096101 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:37.096109 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:37.096113 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:37.096160 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:37.099272 (+ 3112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:37.099887 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:37.099892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:37.099896 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:37.100025 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:37.100387 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:37.100390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:37.100392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:37.100660 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:37.100786 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:37.101175 (+ 389us) server_negotiation.cc:299] Negotiation successful
0810 00:54:37.101393 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 00:54:37.106091 31472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a: Bootstrap starting.
I20260810 00:54:37.109004 31472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a: Neither blocks nor log segments found. Creating new log.
I20260810 00:54:37.109978 31472 log.cc:824] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a: Log is configured to *not* fsync() on all Append() calls
I20260810 00:54:37.112447 31472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a: No bootstrap required, opened a new log
I20260810 00:54:37.121284 31472 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } } peers { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } } peers { permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } }
I20260810 00:54:37.121567 31472 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:54:37.121686 31472 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5d2ef87358e4675a5ba136104201c3a, State: Initialized, Role: FOLLOWER
I20260810 00:54:37.122216 31472 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } } peers { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } } peers { permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } }
I20260810 00:54:37.123068 31644 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } } peers { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } } peers { permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } } }
I20260810 00:54:37.123487 31644 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:54:37.124226 31472 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a [sys.catalog]: configured and running, proceeding with master startup.
W20260810 00:54:37.136685 31293 tablet.cc:2369] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:54:37.141559 31376 master_runner.cc:263] No leader master found from master a5d2ef87358e4675a5ba136104201c3a
I20260810 00:54:37.141691 31376 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 00:54:37.145759 31659 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 00:54:37.145954 31659 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 00:54:37.229312 31628 raft_consensus.cc:482] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:54:37.229614 31628 raft_consensus.cc:504] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } } peers { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } } peers { permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } }
I20260810 00:54:37.231160 31628 leader_election.cc:288] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5d2ef87358e4675a5ba136104201c3a (127.10.97.125:38149), 50ceb7989897418789fbdbe635e0ef8c (127.10.97.124:36497)
I20260810 00:54:37.232350 31547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e424e98e56a945d3bc07e995d514b3fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ceb7989897418789fbdbe635e0ef8c" is_pre_election: true
I20260810 00:54:37.232959 31547 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e424e98e56a945d3bc07e995d514b3fd in term 0.
I20260810 00:54:37.233702 31289 leader_election.cc:302] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ceb7989897418789fbdbe635e0ef8c, e424e98e56a945d3bc07e995d514b3fd; no voters:
I20260810 00:54:37.234890 31438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e424e98e56a945d3bc07e995d514b3fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5d2ef87358e4675a5ba136104201c3a" is_pre_election: true
I20260810 00:54:37.235457 31438 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e424e98e56a945d3bc07e995d514b3fd in term 0.
I20260810 00:54:37.236349 31628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:54:37.236500 31628 raft_consensus.cc:482] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:54:37.236595 31628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:54:37.243105 31628 raft_consensus.cc:504] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } } peers { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } } peers { permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } }
I20260810 00:54:37.245770 31438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e424e98e56a945d3bc07e995d514b3fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5d2ef87358e4675a5ba136104201c3a"
I20260810 00:54:37.245956 31438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:54:37.249476 31438 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e424e98e56a945d3bc07e995d514b3fd in term 1.
I20260810 00:54:37.249909 31284 leader_election.cc:302] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5d2ef87358e4675a5ba136104201c3a, e424e98e56a945d3bc07e995d514b3fd; no voters:
I20260810 00:54:37.252307 31547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e424e98e56a945d3bc07e995d514b3fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ceb7989897418789fbdbe635e0ef8c"
I20260810 00:54:37.252519 31547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:54:37.255892 31547 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e424e98e56a945d3bc07e995d514b3fd in term 1.
I20260810 00:54:37.256460 31628 leader_election.cc:288] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [CANDIDATE]: Term 1 election: Requested vote from peers a5d2ef87358e4675a5ba136104201c3a (127.10.97.125:38149), 50ceb7989897418789fbdbe635e0ef8c (127.10.97.124:36497)
I20260810 00:54:37.256826 31628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:54:37.257210 31628 raft_consensus.cc:686] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [term 1 LEADER]: Becoming Leader. State: Replica: e424e98e56a945d3bc07e995d514b3fd, State: Running, Role: LEADER
I20260810 00:54:37.257982 31628 consensus_queue.cc:237] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } } peers { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } } peers { permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } }
I20260810 00:54:37.269281 31628 sys_catalog.cc:455] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader e424e98e56a945d3bc07e995d514b3fd. Latest consensus state: current_term: 1 leader_uuid: "e424e98e56a945d3bc07e995d514b3fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } } peers { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } } peers { permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } } }
I20260810 00:54:37.271282 31628 sys_catalog.cc:458] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [sys.catalog]: This master's current role is: LEADER
I20260810 00:54:37.278043 31665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:54:37.280426 31665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:54:37.298976 31547 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c [term 1 FOLLOWER]: Refusing update from remote peer e424e98e56a945d3bc07e995d514b3fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:54:37.299991 31628 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:54:37.307255 31438 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a [term 1 FOLLOWER]: Refusing update from remote peer e424e98e56a945d3bc07e995d514b3fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:54:37.308143 31628 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:54:37.345847 31644 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e424e98e56a945d3bc07e995d514b3fd. Latest consensus state: current_term: 1 leader_uuid: "e424e98e56a945d3bc07e995d514b3fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } } peers { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } } peers { permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } } }
I20260810 00:54:37.346197 31644 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:54:37.349908 31666 mvcc.cc:205] Tried to move back new op lower bound from 7316780143791206400 to 7316780143660331008. Current Snapshot: MvccSnapshot[applied={T|T < 7316780143791206400}]
I20260810 00:54:37.355103 31669 mvcc.cc:205] Tried to move back new op lower bound from 7316780143791206400 to 7316780143660331008. Current Snapshot: MvccSnapshot[applied={T|T < 7316780143791206400}]
I20260810 00:54:37.359973 31662 sys_catalog.cc:455] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e424e98e56a945d3bc07e995d514b3fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } } peers { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } } peers { permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } } }
I20260810 00:54:37.360360 31662 sys_catalog.cc:458] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [sys.catalog]: This master's current role is: LEADER
I20260810 00:54:37.363090 31665 catalog_manager.cc:1347] Generated new cluster ID: 9184846985c44368a33b527911cf20fe
I20260810 00:54:37.363197 31665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:54:37.382005 31608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e424e98e56a945d3bc07e995d514b3fd. Latest consensus state: current_term: 1 leader_uuid: "e424e98e56a945d3bc07e995d514b3fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } } peers { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } } peers { permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } } }
I20260810 00:54:37.382315 31608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:54:37.393198 31628 sys_catalog.cc:455] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e424e98e56a945d3bc07e995d514b3fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } } peers { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } } peers { permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } } }
I20260810 00:54:37.394369 31628 sys_catalog.cc:458] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd [sys.catalog]: This master's current role is: LEADER
I20260810 00:54:37.399976 31668 mvcc.cc:205] Tried to move back new op lower bound from 7316780143791206400 to 7316780143660331008. Current Snapshot: MvccSnapshot[applied={T|T < 7316780143791206400}]
I20260810 00:54:37.415959 31608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e424e98e56a945d3bc07e995d514b3fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } } peers { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } } peers { permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } } }
I20260810 00:54:37.416221 31608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:54:37.416519 31644 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e424e98e56a945d3bc07e995d514b3fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e424e98e56a945d3bc07e995d514b3fd" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35089 } } peers { permanent_uuid: "a5d2ef87358e4675a5ba136104201c3a" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 38149 } } peers { permanent_uuid: "50ceb7989897418789fbdbe635e0ef8c" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 36497 } } }
I20260810 00:54:37.416877 31644 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:54:37.429960 31665 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:54:37.431746 31665 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:54:37.450256 31665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e424e98e56a945d3bc07e995d514b3fd: Generated new TSK 0
I20260810 00:54:37.451223 31665 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:54:37.964558 31622 catalog_manager.cc:1259] Loaded cluster ID: 9184846985c44368a33b527911cf20fe
I20260810 00:54:37.964663 31622 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c: loading cluster ID for follower catalog manager: success
I20260810 00:54:37.967237 31622 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c: acquiring CA information for follower catalog manager: success
I20260810 00:54:37.968969 31622 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 50ceb7989897418789fbdbe635e0ef8c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 00:54:38.147902 31659 catalog_manager.cc:1259] Loaded cluster ID: 9184846985c44368a33b527911cf20fe
I20260810 00:54:38.148021 31659 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a: loading cluster ID for follower catalog manager: success
I20260810 00:54:38.150483 31659 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a: acquiring CA information for follower catalog manager: success
I20260810 00:54:38.152314 31659 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a5d2ef87358e4675a5ba136104201c3a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 00:54:38.541021 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 31261
I20260810 00:54:38.542340 31359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:38.647780 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 31261
W20260810 00:54:38.676735 31510 connection.cc:535] server connection from 127.0.0.1:58428 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 00:54:38.677338 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 31376
I20260810 00:54:38.678397 31467 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:38.780752 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 31376
I20260810 00:54:38.806895 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 31484
I20260810 00:54:38.807924 31574 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:38.916134 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 31484
2026-08-10T00:54:38Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4854 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:54:39 dist-test-slave-jlzn krb5kdc[31689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:54:39 dist-test-slave-jlzn krb5kdc[31689](info): set up 2 sockets
Aug 10 00:54:39 dist-test-slave-jlzn krb5kdc[31689](info): commencing operation
krb5kdc: starting...
W20260810 00:54:40.110950 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:54:40 dist-test-slave-jlzn krb5kdc[31689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:54:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:54:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:54:40.338569 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:35145
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:44049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:35145
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:54:40.652318 31714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:40.652577 31714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:40.652714 31714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:40.659595 31714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:54:40.659704 31714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:40.659780 31714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:40.659864 31714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:54:40.659922 31714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:54:40.669124 31714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:44049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:35145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:35145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:40.670257 31714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:40.671470 31714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:40.682353 31725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:40.682395 31731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:40.682430 31727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:40.683207 31714 server_base.cc:1034] running on GCE node
I20260810 00:54:40.683899 31714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:40.685087 31714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:40.686286 31714 hybrid_clock.cc:648] HybridClock initialized: now 1786323280686240 us; error 36 us; skew 500 ppm
I20260810 00:54:40.686569 31714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:40 dist-test-slave-jlzn krb5kdc[31689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323280, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:40.689505 31714 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:54:40.690793 31714 webserver.cc:460] Webserver started at http://127.10.97.126:35305/ using document root <none> and password file <none>
I20260810 00:54:40.691380 31714 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:40.691478 31714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:40.691733 31714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:54:40.694489 31714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "0797d017353c416b87f275653c08106d"
format_stamp: "Formatted at 2026-08-10 00:54:40 on dist-test-slave-jlzn"
server_key: "fe7ac6d5cb58fc02acc3648659873dbd"
server_key_iv: "7b85fbd5afd6e48f9d2a4e82887232f6"
server_key_version: "encryptionkey@0"
I20260810 00:54:40.695159 31714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "0797d017353c416b87f275653c08106d"
format_stamp: "Formatted at 2026-08-10 00:54:40 on dist-test-slave-jlzn"
server_key: "fe7ac6d5cb58fc02acc3648659873dbd"
server_key_iv: "7b85fbd5afd6e48f9d2a4e82887232f6"
server_key_version: "encryptionkey@0"
I20260810 00:54:40.699774 31714 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260810 00:54:40.702889 31741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:40.704094 31714 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:54:40.704238 31714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "0797d017353c416b87f275653c08106d"
format_stamp: "Formatted at 2026-08-10 00:54:40 on dist-test-slave-jlzn"
server_key: "fe7ac6d5cb58fc02acc3648659873dbd"
server_key_iv: "7b85fbd5afd6e48f9d2a4e82887232f6"
server_key_version: "encryptionkey@0"
I20260810 00:54:40.704391 31714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:40.739527 31714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:40.743595 31714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:40.743851 31714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:40.760380 31714 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:35145
I20260810 00:54:40.760402 31815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:35145 every 8 connection(s)
I20260810 00:54:40.761672 31714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:54:40.765165 31817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:54:40.766938 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 31714
I20260810 00:54:40.767156 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:54:40.767849 10629 external_mini_cluster.cc:1428] Setting key d450ecffe172d62886e94eac73ad1797
I20260810 00:54:40.776433 31817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d: Bootstrap starting.
I20260810 00:54:40.779959 31817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d: Neither blocks nor log segments found. Creating new log.
I20260810 00:54:40.781198 31817 log.cc:824] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d: Log is configured to *not* fsync() on all Append() calls
Aug 10 00:54:40 dist-test-slave-jlzn krb5kdc[31689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:54:40.783952 31817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d: No bootstrap required, opened a new log
I20260810 00:54:40.790405 31820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:40.769655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42032 (local address 127.10.97.126:35145)
0810 00:54:40.770457 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:40.770469 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:40.770511 (+ 42us) server_negotiation.cc:407] Connection header received
0810 00:54:40.772238 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:40.772261 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:40.772624 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:40.772930 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:40.774692 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:40.775980 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:40.777800 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:40.778470 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:40.778624 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:40.783349 (+ 4725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:40.783356 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:40.783370 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:40.783408 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:40.786565 (+ 3157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:40.787234 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:40.787240 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:40.787246 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:40.787416 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:40.787789 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:40.787793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:40.787795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:40.788345 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:40.788646 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:40.789076 (+ 430us) server_negotiation.cc:299] Negotiation successful
0810 00:54:40.789466 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":364,"threads_started":1}
I20260810 00:54:40.791656 31817 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0797d017353c416b87f275653c08106d" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35145 } }
I20260810 00:54:40.792042 31817 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:54:40.792141 31817 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0797d017353c416b87f275653c08106d, State: Initialized, Role: FOLLOWER
I20260810 00:54:40.792915 31817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0797d017353c416b87f275653c08106d" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35145 } }
I20260810 00:54:40.793172 31817 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:54:40.793318 31817 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:54:40.793500 31817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:54:40.794200 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:54:40.795713 31817 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0797d017353c416b87f275653c08106d" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35145 } }
I20260810 00:54:40.796202 31817 leader_election.cc:302] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0797d017353c416b87f275653c08106d; no voters:
I20260810 00:54:40.796859 31817 leader_election.cc:288] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:54:40.796972 31825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:54:40.797762 31825 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d [term 1 LEADER]: Becoming Leader. State: Replica: 0797d017353c416b87f275653c08106d, State: Running, Role: LEADER
I20260810 00:54:40.798229 31825 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0797d017353c416b87f275653c08106d" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35145 } }
I20260810 00:54:40.798804 31817 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:54:40.800896 31826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0797d017353c416b87f275653c08106d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0797d017353c416b87f275653c08106d" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35145 } } }
I20260810 00:54:40.801311 31826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d [sys.catalog]: This master's current role is: LEADER
I20260810 00:54:40.800873 31828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0797d017353c416b87f275653c08106d. Latest consensus state: current_term: 1 leader_uuid: "0797d017353c416b87f275653c08106d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0797d017353c416b87f275653c08106d" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 35145 } } }
I20260810 00:54:40.801661 31828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d [sys.catalog]: This master's current role is: LEADER
I20260810 00:54:40.802902 31836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:54:40.807123 31836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:54:40.815397 31836 catalog_manager.cc:1347] Generated new cluster ID: f5d370725c9745269e6e882ec5e72874
I20260810 00:54:40.815501 31836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:54:40.828670 31836 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:54:40.830518 31836 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:54:40.838982 31836 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0797d017353c416b87f275653c08106d: Generated new TSK 0
I20260810 00:54:40.840085 31836 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:54:40.854848 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 31714
I20260810 00:54:40.856000 31811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:40.945101 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 31714
2026-08-10T00:54:40Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2029 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:54:41 dist-test-slave-jlzn krb5kdc[31862](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:54:41 dist-test-slave-jlzn krb5kdc[31862](info): set up 2 sockets
Aug 10 00:54:41 dist-test-slave-jlzn krb5kdc[31862](info): commencing operation
krb5kdc: starting...
W20260810 00:54:42.132196 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:54:42 dist-test-slave-jlzn krb5kdc[31862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:54:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:54:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:54:42.358861 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:43387
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:43387
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:54:42.673355 31886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:42.673638 31886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:42.673771 31886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:42.680709 31886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:54:42.680819 31886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:42.680895 31886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:42.680974 31886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:54:42.681030 31886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:54:42.690433 31886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:43387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:43387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:42.691607 31886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:42.692873 31886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:42.703951 31897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:42.703927 31902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:42.703893 31899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:42.704948 31886 server_base.cc:1034] running on GCE node
I20260810 00:54:42.705669 31886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:42.706908 31886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:42.708109 31886 hybrid_clock.cc:648] HybridClock initialized: now 1786323282708060 us; error 30 us; skew 500 ppm
I20260810 00:54:42.708418 31886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:42 dist-test-slave-jlzn krb5kdc[31862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323282, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:42.711328 31886 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:54:42.712800 31886 webserver.cc:460] Webserver started at http://127.10.97.126:36949/ using document root <none> and password file <none>
I20260810 00:54:42.713410 31886 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:42.713534 31886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:42.713809 31886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:54:42.716643 31886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "0cd8f87eb6cc40dd877cda931a8498db"
format_stamp: "Formatted at 2026-08-10 00:54:42 on dist-test-slave-jlzn"
server_key: "9c1d78c30d6bdd8284defd7afb2af23d"
server_key_iv: "1af3faefcd158fa099a3107e7639725f"
server_key_version: "encryptionkey@0"
I20260810 00:54:42.717361 31886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "0cd8f87eb6cc40dd877cda931a8498db"
format_stamp: "Formatted at 2026-08-10 00:54:42 on dist-test-slave-jlzn"
server_key: "9c1d78c30d6bdd8284defd7afb2af23d"
server_key_iv: "1af3faefcd158fa099a3107e7639725f"
server_key_version: "encryptionkey@0"
I20260810 00:54:42.722142 31886 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:54:42.725358 31916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:42.726541 31886 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:54:42.726699 31886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "0cd8f87eb6cc40dd877cda931a8498db"
format_stamp: "Formatted at 2026-08-10 00:54:42 on dist-test-slave-jlzn"
server_key: "9c1d78c30d6bdd8284defd7afb2af23d"
server_key_iv: "1af3faefcd158fa099a3107e7639725f"
server_key_version: "encryptionkey@0"
I20260810 00:54:42.726864 31886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:42.744316 31886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:42.747843 31886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:42.748088 31886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:42.764679 31886 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:43387
I20260810 00:54:42.764722 31983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:43387 every 8 connection(s)
I20260810 00:54:42.765864 31886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:54:42.767103 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 31886
I20260810 00:54:42.767356 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:54:42.768219 10629 external_mini_cluster.cc:1428] Setting key b63752e92741f7a8aef4d750d100d817
I20260810 00:54:42.770088 31984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 00:54:42 dist-test-slave-jlzn krb5kdc[31862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:54:42.782847 31984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db: Bootstrap starting.
I20260810 00:54:42.786644 31984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db: Neither blocks nor log segments found. Creating new log.
I20260810 00:54:42.787818 31984 log.cc:824] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db: Log is configured to *not* fsync() on all Append() calls
I20260810 00:54:42.790772 31984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db: No bootstrap required, opened a new log
I20260810 00:54:42.790819 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:42.770128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38924 (local address 127.10.97.126:43387)
0810 00:54:42.770847 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:42.770859 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:42.770897 (+ 38us) server_negotiation.cc:407] Connection header received
0810 00:54:42.772473 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:42.772503 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:42.772850 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:42.773149 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:42.774800 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:42.776146 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:42.778075 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:42.778722 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:42.778862 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:42.783336 (+ 4474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:42.783346 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:42.783363 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:42.783405 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:42.787062 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:42.787694 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:42.787702 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:42.787709 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:42.787949 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:42.788341 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:42.788347 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:42.788351 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:42.788876 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:42.789137 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:42.789577 (+ 440us) server_negotiation.cc:299] Negotiation successful
0810 00:54:42.789883 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":316,"threads_started":1}
I20260810 00:54:42.798170 31984 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd8f87eb6cc40dd877cda931a8498db" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43387 } }
I20260810 00:54:42.798481 31984 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:54:42.798566 31984 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd8f87eb6cc40dd877cda931a8498db, State: Initialized, Role: FOLLOWER
I20260810 00:54:42.799117 31984 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd8f87eb6cc40dd877cda931a8498db" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43387 } }
I20260810 00:54:42.799418 31984 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:54:42.799556 31984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:54:42.799741 31984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:54:42.801781 31984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd8f87eb6cc40dd877cda931a8498db" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43387 } }
I20260810 00:54:42.802235 31984 leader_election.cc:302] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cd8f87eb6cc40dd877cda931a8498db; no voters:
I20260810 00:54:42.802763 31984 leader_election.cc:288] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:54:42.802872 31990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:54:42.803462 31990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 1 LEADER]: Becoming Leader. State: Replica: 0cd8f87eb6cc40dd877cda931a8498db, State: Running, Role: LEADER
I20260810 00:54:42.804013 31990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd8f87eb6cc40dd877cda931a8498db" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43387 } }
I20260810 00:54:42.804627 31984 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:54:42.806697 31992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cd8f87eb6cc40dd877cda931a8498db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd8f87eb6cc40dd877cda931a8498db" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43387 } } }
I20260810 00:54:42.807032 31992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [sys.catalog]: This master's current role is: LEADER
I20260810 00:54:42.807333 31995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cd8f87eb6cc40dd877cda931a8498db. Latest consensus state: current_term: 1 leader_uuid: "0cd8f87eb6cc40dd877cda931a8498db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd8f87eb6cc40dd877cda931a8498db" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43387 } } }
I20260810 00:54:42.807673 31995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [sys.catalog]: This master's current role is: LEADER
I20260810 00:54:42.812812 32020 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 00:54:42.812985 32007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:54:42.817082 32007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:54:42.824697 32007 catalog_manager.cc:1347] Generated new cluster ID: fa63f84ed5a1400686fb322b339b7056
I20260810 00:54:42.824793 32007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:54:42.843969 32007 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:54:42.845196 32007 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:54:42.854579 32007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db: Generated new TSK 0
I20260810 00:54:42.855304 32007 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
I20260810 00:54:42.915356 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:43387
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:54:43.230880 32027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:43.231158 32027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:43.231292 32027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:43.238289 32027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:43.238379 32027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:43.238523 32027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:54:43.247474 32027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:43387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:43.248579 32027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:43.249758 32027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:43.261922 32034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:43.261963 32037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:43.262023 32035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:43.262673 32027 server_base.cc:1034] running on GCE node
I20260810 00:54:43.263239 32027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:43.264205 32027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:43.265391 32027 hybrid_clock.cc:648] HybridClock initialized: now 1786323283265351 us; error 35 us; skew 500 ppm
I20260810 00:54:43.265664 32027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:43 dist-test-slave-jlzn krb5kdc[31862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323283, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:43.268680 32027 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:54:43.269968 32027 webserver.cc:460] Webserver started at http://127.10.97.65:43789/ using document root <none> and password file <none>
I20260810 00:54:43.270582 32027 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:43.270699 32027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:43.270959 32027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:54:43.273681 32027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "101a6277a18a47aaa8e2a0c52a7cd1da"
format_stamp: "Formatted at 2026-08-10 00:54:43 on dist-test-slave-jlzn"
server_key: "3afb52af1450e397d5818f94648093de"
server_key_iv: "fc629ac818c3b9b356144d02f518ef96"
server_key_version: "encryptionkey@0"
I20260810 00:54:43.274422 32027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "101a6277a18a47aaa8e2a0c52a7cd1da"
format_stamp: "Formatted at 2026-08-10 00:54:43 on dist-test-slave-jlzn"
server_key: "3afb52af1450e397d5818f94648093de"
server_key_iv: "fc629ac818c3b9b356144d02f518ef96"
server_key_version: "encryptionkey@0"
I20260810 00:54:43.279246 32027 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260810 00:54:43.282439 32050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:43.283623 32027 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:54:43.283773 32027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "101a6277a18a47aaa8e2a0c52a7cd1da"
format_stamp: "Formatted at 2026-08-10 00:54:43 on dist-test-slave-jlzn"
server_key: "3afb52af1450e397d5818f94648093de"
server_key_iv: "fc629ac818c3b9b356144d02f518ef96"
server_key_version: "encryptionkey@0"
I20260810 00:54:43.283975 32027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:43.304697 32027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:43.308772 32027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:43.309020 32027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:43.309950 32027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:54:43.311168 32027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:54:43.311245 32027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:43.311364 32027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:54:43.311415 32027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:43.340853 32027 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:42033
I20260810 00:54:43.340911 32225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:42033 every 8 connection(s)
I20260810 00:54:43.341980 32027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:54:43.347316 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 32027
I20260810 00:54:43.347502 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:54:43.348176 10629 external_mini_cluster.cc:1428] Setting key 10d178853e7ac9bdffaba5be4eaab9f4
Aug 10 00:54:43 dist-test-slave-jlzn krb5kdc[31862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323283, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:54:43.360222 32230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:43.344223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:51629 (local address 127.10.97.126:43387)
0810 00:54:43.344675 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:43.344681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:43.345520 (+ 839us) server_negotiation.cc:407] Connection header received
0810 00:54:43.347195 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:43.347201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:43.347283 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:43.347379 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:43.348885 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:43.349662 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:43.351139 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:43.351538 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:43.351655 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:43.354885 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:43.354890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:43.354893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:43.354924 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:43.357413 (+ 2489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:43.358073 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:43.358077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:43.358080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:43.358200 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:43.358589 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:43.358594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:43.358597 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:43.358864 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:43.358995 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:43.359614 (+ 619us) server_negotiation.cc:299] Negotiation successful
0810 00:54:43.359879 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":206,"threads_started":1}
I20260810 00:54:43.361006 32231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:43.344691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:51629)
0810 00:54:43.345363 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:43.345406 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:43.346932 (+ 1526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:43.347630 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:43.347640 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:43.348152 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:43.348712 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:43.348731 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:43.349807 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:43.349810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:43.350994 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:43.351007 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:43.351755 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:43.351763 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:43.352010 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:43.352589 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:43.352615 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:43.354670 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:43.357540 (+ 2870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:43.357549 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:43.357563 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:43.357961 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:43.358304 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:43.358309 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:43.358312 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:43.358484 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:43.358987 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:43.358994 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:43.359287 (+ 293us) client_negotiation.cc:769] Sending connection context
0810 00:54:43.359510 (+ 223us) client_negotiation.cc:240] Negotiation successful
0810 00:54:43.359987 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":248,"threads_started":1}
I20260810 00:54:43.362365 32226 heartbeater.cc:347] Connected to a master server at 127.10.97.126:43387
I20260810 00:54:43.362650 32226 heartbeater.cc:464] Registering TS with master...
I20260810 00:54:43.363495 32226 heartbeater.cc:511] Master 127.10.97.126:43387 requested a full tablet report, sending...
I20260810 00:54:43.365636 31936 ts_manager.cc:194] Registered new tserver with Master: 101a6277a18a47aaa8e2a0c52a7cd1da (127.10.97.65:42033)
I20260810 00:54:43.367506 31936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:51629
WARNING: no policy specified for kudu/127.10.97.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
I20260810 00:54:43.404517 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:43387
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:54:43.714823 32235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:43.715070 32235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:43.715195 32235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:43.722419 32235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:43.722541 32235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:43.722678 32235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:54:43.731945 32235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:43387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:43.733112 32235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:43.734315 32235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:43.746513 32240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:43.746548 32243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:43.746472 32241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:43.747316 32235 server_base.cc:1034] running on GCE node
I20260810 00:54:43.747896 32235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:43.748750 32235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:43.749944 32235 hybrid_clock.cc:648] HybridClock initialized: now 1786323283749890 us; error 45 us; skew 500 ppm
I20260810 00:54:43.750226 32235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:43 dist-test-slave-jlzn krb5kdc[31862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323283, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:43.753214 32235 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:54:43.754472 32235 webserver.cc:460] Webserver started at http://127.10.97.66:35189/ using document root <none> and password file <none>
I20260810 00:54:43.755084 32235 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:43.755204 32235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:43.755478 32235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:54:43.758215 32235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "90ed6faf3425442ab2a6249ee33a62c8"
format_stamp: "Formatted at 2026-08-10 00:54:43 on dist-test-slave-jlzn"
server_key: "e080b34aa8b3334c315c0756d892a049"
server_key_iv: "0c529fcc224024c89a889c99edef76f8"
server_key_version: "encryptionkey@0"
I20260810 00:54:43.758953 32235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "90ed6faf3425442ab2a6249ee33a62c8"
format_stamp: "Formatted at 2026-08-10 00:54:43 on dist-test-slave-jlzn"
server_key: "e080b34aa8b3334c315c0756d892a049"
server_key_iv: "0c529fcc224024c89a889c99edef76f8"
server_key_version: "encryptionkey@0"
I20260810 00:54:43.763610 32235 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260810 00:54:43.766959 32254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:43.768285 32235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:54:43.768435 32235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "90ed6faf3425442ab2a6249ee33a62c8"
format_stamp: "Formatted at 2026-08-10 00:54:43 on dist-test-slave-jlzn"
server_key: "e080b34aa8b3334c315c0756d892a049"
server_key_iv: "0c529fcc224024c89a889c99edef76f8"
server_key_version: "encryptionkey@0"
I20260810 00:54:43.768605 32235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:43.801584 32235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:43.805321 32235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:43.805586 32235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:43.806504 32235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:54:43.807734 32235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:54:43.807825 32235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:43.807953 32235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:54:43.808003 32235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:43.837568 32235 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:39415
I20260810 00:54:43.837613 32434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:39415 every 8 connection(s)
I20260810 00:54:43.838730 32235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:54:43.844344 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 32235
I20260810 00:54:43.844549 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:54:43.845003 10629 external_mini_cluster.cc:1428] Setting key caaa9960829919661b762d7cf2b88a63
Aug 10 00:54:43 dist-test-slave-jlzn krb5kdc[31862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323283, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:54:43.857401 32230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:43.841032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:54017 (local address 127.10.97.126:43387)
0810 00:54:43.841234 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:43.841241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:43.842282 (+ 1041us) server_negotiation.cc:407] Connection header received
0810 00:54:43.844019 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:43.844025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:43.844147 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:43.844255 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:43.845683 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:43.846722 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:43.848286 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:43.848706 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:43.848814 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:43.852114 (+ 3300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:43.852119 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:43.852122 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:43.852154 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:43.854557 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:43.855214 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:43.855219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:43.855221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:43.855357 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:43.855759 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:43.855762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:43.855764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:43.856050 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:43.856184 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:43.856826 (+ 642us) server_negotiation.cc:299] Negotiation successful
0810 00:54:43.857068 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 00:54:43.858268 32437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:43.841474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:54017)
0810 00:54:43.842127 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:43.842171 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:43.843720 (+ 1549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:43.844421 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:43.844429 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:43.844942 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:43.845509 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:43.845526 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:43.846858 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:43.846861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:43.848150 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:43.848162 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:43.848922 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:43.848929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:43.849144 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:43.849731 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:43.849757 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:43.851916 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:43.854685 (+ 2769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:43.854694 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:43.854707 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:43.855090 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:43.855459 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:43.855464 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:43.855468 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:43.855648 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:43.856175 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:43.856182 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:43.856473 (+ 291us) client_negotiation.cc:769] Sending connection context
0810 00:54:43.856701 (+ 228us) client_negotiation.cc:240] Negotiation successful
0810 00:54:43.857173 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":245,"threads_started":1}
I20260810 00:54:43.859505 32435 heartbeater.cc:347] Connected to a master server at 127.10.97.126:43387
I20260810 00:54:43.859771 32435 heartbeater.cc:464] Registering TS with master...
I20260810 00:54:43.860450 32435 heartbeater.cc:511] Master 127.10.97.126:43387 requested a full tablet report, sending...
I20260810 00:54:43.861793 31936 ts_manager.cc:194] Registered new tserver with Master: 90ed6faf3425442ab2a6249ee33a62c8 (127.10.97.66:39415)
I20260810 00:54:43.862589 31936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:54017
WARNING: no policy specified for kudu/127.10.97.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
I20260810 00:54:43.898684 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:43387
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:54:44.212803 32442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:44.213071 32442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:44.213176 32442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:44.219988 32442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:44.220077 32442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:44.220193 32442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:54:44.229199 32442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:43387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:44.230345 32442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:44.231607 32442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:44.243597 32453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:44.243664 32449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:44.243688 32448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:44.244885 32442 server_base.cc:1034] running on GCE node
I20260810 00:54:44.245457 32442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:44.246286 32442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:44.247481 32442 hybrid_clock.cc:648] HybridClock initialized: now 1786323284247440 us; error 29 us; skew 500 ppm
I20260810 00:54:44.247759 32442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:44 dist-test-slave-jlzn krb5kdc[31862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323284, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:44.250664 32442 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:54:44.252046 32442 webserver.cc:460] Webserver started at http://127.10.97.67:46697/ using document root <none> and password file <none>
I20260810 00:54:44.252681 32442 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:44.252789 32442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:44.253058 32442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:54:44.255687 32442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "d833f3e4ccaa4e329cfec01b259b12b5"
format_stamp: "Formatted at 2026-08-10 00:54:44 on dist-test-slave-jlzn"
server_key: "4375b6f8ffe702189e57a4a0feb428a2"
server_key_iv: "63cf1f9f1cbe16c4534ec62f3a7061d1"
server_key_version: "encryptionkey@0"
I20260810 00:54:44.256440 32442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "d833f3e4ccaa4e329cfec01b259b12b5"
format_stamp: "Formatted at 2026-08-10 00:54:44 on dist-test-slave-jlzn"
server_key: "4375b6f8ffe702189e57a4a0feb428a2"
server_key_iv: "63cf1f9f1cbe16c4534ec62f3a7061d1"
server_key_version: "encryptionkey@0"
I20260810 00:54:44.261034 32442 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260810 00:54:44.264119 32465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:44.265266 32442 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260810 00:54:44.265408 32442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "d833f3e4ccaa4e329cfec01b259b12b5"
format_stamp: "Formatted at 2026-08-10 00:54:44 on dist-test-slave-jlzn"
server_key: "4375b6f8ffe702189e57a4a0feb428a2"
server_key_iv: "63cf1f9f1cbe16c4534ec62f3a7061d1"
server_key_version: "encryptionkey@0"
I20260810 00:54:44.265591 32442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:44.278754 32442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:44.282244 32442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:44.282490 32442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:44.283316 32442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:54:44.284544 32442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:54:44.284616 32442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:44.284742 32442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:54:44.284798 32442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:44.314399 32442 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:34171
I20260810 00:54:44.314445 32639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:34171 every 8 connection(s)
I20260810 00:54:44.315551 32442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:54:44.319387 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 32442
I20260810 00:54:44.319557 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:54:44.319994 10629 external_mini_cluster.cc:1428] Setting key 695f9cd2d5cd2832b47d8e8ad49e0288
Aug 10 00:54:44 dist-test-slave-jlzn krb5kdc[31862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323284, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:54:44.333835 32230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.317827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:49189 (local address 127.10.97.126:43387)
0810 00:54:44.317988 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:44.317993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:44.319027 (+ 1034us) server_negotiation.cc:407] Connection header received
0810 00:54:44.320934 (+ 1907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:44.320940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:44.321025 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:44.321106 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:44.322485 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.323263 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.324852 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.325266 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.325371 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:44.328465 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:44.328469 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:44.328472 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:44.328503 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:44.331035 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:44.331692 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:44.331696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:44.331698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:44.331842 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:44.332254 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:44.332257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:44.332259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:44.332516 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:44.332654 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:44.333258 (+ 604us) server_negotiation.cc:299] Negotiation successful
0810 00:54:44.333503 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 00:54:44.334609 32642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.318231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:49189)
0810 00:54:44.318874 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:44.318914 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:44.320690 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:44.321338 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:44.321346 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:44.321804 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:44.322331 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:44.322348 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.323392 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.323395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:44.324722 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:44.324735 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.325468 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.325476 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:44.325687 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:44.326257 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:44.326278 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:44.328282 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:44.331168 (+ 2886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:44.331176 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:44.331190 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:44.331570 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:44.331948 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:44.331953 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:44.331957 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:44.332142 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:44.332635 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:44.332643 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:44.332925 (+ 282us) client_negotiation.cc:769] Sending connection context
0810 00:54:44.333141 (+ 216us) client_negotiation.cc:240] Negotiation successful
0810 00:54:44.333582 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":260,"threads_started":1}
I20260810 00:54:44.335856 32640 heartbeater.cc:347] Connected to a master server at 127.10.97.126:43387
I20260810 00:54:44.336107 32640 heartbeater.cc:464] Registering TS with master...
I20260810 00:54:44.336717 32640 heartbeater.cc:511] Master 127.10.97.126:43387 requested a full tablet report, sending...
I20260810 00:54:44.337945 31936 ts_manager.cc:194] Registered new tserver with Master: d833f3e4ccaa4e329cfec01b259b12b5 (127.10.97.67:34171)
I20260810 00:54:44.338717 31936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:49189
I20260810 00:54:44.346462 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:54:44.361549 32230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.350602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38926 (local address 127.10.97.126:43387)
0810 00:54:44.350742 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:44.350746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:44.351181 (+ 435us) server_negotiation.cc:407] Connection header received
0810 00:54:44.351303 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:44.351307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:44.351357 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:44.351429 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:44.352757 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.353423 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.354621 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.355000 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.355097 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:44.357157 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:44.357162 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:44.357165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:44.357191 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:44.359497 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:44.360085 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:44.360088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:44.360091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:44.360206 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:44.360491 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:44.360493 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:44.360495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:44.360772 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:44.360936 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:44.361199 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 00:54:44.361342 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 00:54:44.365046 31936 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 00:54:44.370627 32226 heartbeater.cc:503] Master 127.10.97.126:43387 was elected leader, sending a full tablet report...
W20260810 00:54:44.371882 31936 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:54:44.393746 32654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.382775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36308 (local address 127.10.97.65:42033)
0810 00:54:44.385730 (+ 2955us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:44.385736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:44.385765 (+ 29us) server_negotiation.cc:407] Connection header received
0810 00:54:44.385832 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:44.385837 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:44.386002 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:44.386169 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:54:44.386981 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.388736 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.391140 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.392605 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.392759 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:44.392985 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:44.393270 (+ 285us) server_negotiation.cc:299] Negotiation successful
0810 00:54:44.393427 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2806,"thread_start_us":2666,"threads_started":1}
I20260810 00:54:44.393795 32653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.382463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.0.0.1:36308)
0810 00:54:44.383103 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:44.383146 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:44.383306 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:44.386191 (+ 2885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:44.386198 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:44.386225 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:54:44.386559 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:44.386572 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.388867 (+ 2295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.388873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:44.390954 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:44.390967 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.392818 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.392824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:44.393006 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:44.393062 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 00:54:44.393207 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 00:54:44.393374 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":43,"thread_start_us":258,"threads_started":1}
I20260810 00:54:44.393958 32655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.383129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:39415 (local address 127.0.0.1:52782)
0810 00:54:44.384820 (+ 1691us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:44.384838 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:44.384951 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:44.387377 (+ 2426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:44.387381 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:44.387404 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:54:44.387796 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:44.387829 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.389582 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.389588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:44.391605 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:44.391616 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.393106 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.393112 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:44.393351 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:44.393386 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 00:54:44.393469 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 00:54:44.393587 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1530,"mutex_wait_us":43,"thread_start_us":1453,"threads_started":1}
I20260810 00:54:44.394374 32656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.383501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:34171 (local address 127.0.0.1:42826)
0810 00:54:44.385377 (+ 1876us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:44.385394 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:44.385495 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:44.386743 (+ 1248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:44.386748 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:44.386772 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:54:44.387214 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:44.387229 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.389056 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.389062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:44.391235 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:44.391251 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.393151 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.393156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:44.393374 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:44.393409 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 00:54:44.393497 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 00:54:44.393622 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1782,"mutex_wait_us":47,"thread_start_us":1667,"threads_started":1}
I20260810 00:54:44.394730 32657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.383686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52782 (local address 127.10.97.66:39415)
0810 00:54:44.386739 (+ 3053us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:44.386747 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:44.386770 (+ 23us) server_negotiation.cc:407] Connection header received
0810 00:54:44.386858 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:44.386864 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:44.387095 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:44.387968 (+ 873us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:54:44.388160 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.389442 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.391741 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.392654 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.392775 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:44.392953 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:44.394289 (+ 1336us) server_negotiation.cc:299] Negotiation successful
0810 00:54:44.394454 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2906,"thread_start_us":2805,"threads_started":1}
I20260810 00:54:44.394353 32660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.384108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42826 (local address 127.10.97.67:34171)
0810 00:54:44.386083 (+ 1975us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:44.386091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:44.386122 (+ 31us) server_negotiation.cc:407] Connection header received
0810 00:54:44.386219 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:44.386226 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:44.386430 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:44.386612 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:54:44.387437 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.388938 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.391409 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.392736 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.392839 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:44.393015 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:44.393895 (+ 880us) server_negotiation.cc:299] Negotiation successful
0810 00:54:44.394035 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1816,"thread_start_us":1671,"threads_started":1}
I20260810 00:54:44.397706 32331 tablet_service.cc:1467] Processing CreateTablet for tablet 1dba3e4ca651433b84289229da86927d (DEFAULT_TABLE table=test-table [id=39ed9b7a96554e1baa672fcac97b0bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:54:44.397775 32124 tablet_service.cc:1467] Processing CreateTablet for tablet 1dba3e4ca651433b84289229da86927d (DEFAULT_TABLE table=test-table [id=39ed9b7a96554e1baa672fcac97b0bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:54:44.398859 32331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dba3e4ca651433b84289229da86927d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:54:44.398633 32538 tablet_service.cc:1467] Processing CreateTablet for tablet 1dba3e4ca651433b84289229da86927d (DEFAULT_TABLE table=test-table [id=39ed9b7a96554e1baa672fcac97b0bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:54:44.399154 32124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dba3e4ca651433b84289229da86927d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:54:44.399909 32538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dba3e4ca651433b84289229da86927d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:54:44.409559 32661 tablet_bootstrap.cc:492] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8: Bootstrap starting.
I20260810 00:54:44.412717 32662 tablet_bootstrap.cc:492] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da: Bootstrap starting.
I20260810 00:54:44.413901 32661 tablet_bootstrap.cc:654] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8: Neither blocks nor log segments found. Creating new log.
I20260810 00:54:44.415129 32661 log.cc:824] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8: Log is configured to *not* fsync() on all Append() calls
I20260810 00:54:44.416045 32662 tablet_bootstrap.cc:654] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da: Neither blocks nor log segments found. Creating new log.
I20260810 00:54:44.417301 32662 log.cc:824] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da: Log is configured to *not* fsync() on all Append() calls
I20260810 00:54:44.418048 32664 tablet_bootstrap.cc:492] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5: Bootstrap starting.
I20260810 00:54:44.418180 32661 tablet_bootstrap.cc:492] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8: No bootstrap required, opened a new log
I20260810 00:54:44.418431 32661 ts_tablet_manager.cc:1397] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 00:54:44.419661 32662 tablet_bootstrap.cc:492] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da: No bootstrap required, opened a new log
I20260810 00:54:44.419925 32662 ts_tablet_manager.cc:1397] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260810 00:54:44.421309 32664 tablet_bootstrap.cc:654] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5: Neither blocks nor log segments found. Creating new log.
I20260810 00:54:44.422497 32664 log.cc:824] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5: Log is configured to *not* fsync() on all Append() calls
I20260810 00:54:44.425287 32664 tablet_bootstrap.cc:492] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5: No bootstrap required, opened a new log
I20260810 00:54:44.425534 32664 ts_tablet_manager.cc:1397] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260810 00:54:44.427018 32662 raft_consensus.cc:348] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } }
I20260810 00:54:44.427326 32662 raft_consensus.cc:374] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:54:44.427423 32662 raft_consensus.cc:729] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101a6277a18a47aaa8e2a0c52a7cd1da, State: Initialized, Role: FOLLOWER
I20260810 00:54:44.427825 32661 raft_consensus.cc:348] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } }
I20260810 00:54:44.427978 32662 consensus_queue.cc:260] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } }
I20260810 00:54:44.428211 32661 raft_consensus.cc:374] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:54:44.428349 32661 raft_consensus.cc:729] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90ed6faf3425442ab2a6249ee33a62c8, State: Initialized, Role: FOLLOWER
I20260810 00:54:44.428989 32661 consensus_queue.cc:260] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } }
I20260810 00:54:44.430111 32435 heartbeater.cc:503] Master 127.10.97.126:43387 was elected leader, sending a full tablet report...
I20260810 00:54:44.431965 32661 ts_tablet_manager.cc:1428] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260810 00:54:44.433455 32662 ts_tablet_manager.cc:1428] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260810 00:54:44.434887 32664 raft_consensus.cc:348] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } }
I20260810 00:54:44.435273 32664 raft_consensus.cc:374] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:54:44.435384 32664 raft_consensus.cc:729] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d833f3e4ccaa4e329cfec01b259b12b5, State: Initialized, Role: FOLLOWER
I20260810 00:54:44.435958 32664 consensus_queue.cc:260] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } }
I20260810 00:54:44.437093 32640 heartbeater.cc:503] Master 127.10.97.126:43387 was elected leader, sending a full tablet report...
I20260810 00:54:44.437752 32664 ts_tablet_manager.cc:1428] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5: Time spent starting tablet: real 0.012s user 0.006s sys 0.007s
I20260810 00:54:44.519349 32672 raft_consensus.cc:482] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:54:44.519670 32672 raft_consensus.cc:504] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } }
I20260810 00:54:44.521258 32672 leader_election.cc:288] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 101a6277a18a47aaa8e2a0c52a7cd1da (127.10.97.65:42033), d833f3e4ccaa4e329cfec01b259b12b5 (127.10.97.67:34171)
I20260810 00:54:44.528841 32654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.521689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:42393 (local address 127.10.97.65:42033)
0810 00:54:44.521909 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:44.521916 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:44.522304 (+ 388us) server_negotiation.cc:407] Connection header received
0810 00:54:44.522534 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:44.522541 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:44.522635 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:44.522725 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:54:44.523339 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.524331 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.526753 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.527936 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.528088 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:44.528161 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:44.528500 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 00:54:44.528598 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 00:54:44.529006 32674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.521605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.10.97.66:42393)
0810 00:54:44.522201 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:44.522217 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:44.522360 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:44.522782 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:44.522786 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:44.522814 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:54:44.523215 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:44.523225 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.524445 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.524449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:44.526607 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:44.526621 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.528122 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.528126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:44.528336 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:44.528376 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 00:54:44.528473 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 00:54:44.528596 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":73,"thread_start_us":318,"threads_started":1}
I20260810 00:54:44.529035 32660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.522482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:56135 (local address 127.10.97.67:34171)
0810 00:54:44.522688 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:44.522693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:44.523015 (+ 322us) server_negotiation.cc:407] Connection header received
0810 00:54:44.523176 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:44.523179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:44.523238 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:44.523318 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:54:44.523780 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.524732 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.526960 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.528160 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.528301 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:44.528361 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:44.528694 (+ 333us) server_negotiation.cc:299] Negotiation successful
0810 00:54:44.528814 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 00:54:44.529109 32675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.522330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:34171 (local address 127.10.97.66:56135)
0810 00:54:44.522906 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:44.522927 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:44.523051 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:44.523344 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:44.523347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:44.523366 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:54:44.523626 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:44.523641 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.524860 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.524865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:44.526833 (+ 1968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:44.526843 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.528370 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.528375 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:44.528546 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:44.528576 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 00:54:44.528662 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 00:54:44.528773 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":24,"thread_start_us":254,"threads_started":1}
I20260810 00:54:44.530042 32153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dba3e4ca651433b84289229da86927d" candidate_uuid: "90ed6faf3425442ab2a6249ee33a62c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" is_pre_election: true
I20260810 00:54:44.530100 32566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dba3e4ca651433b84289229da86927d" candidate_uuid: "90ed6faf3425442ab2a6249ee33a62c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" is_pre_election: true
I20260810 00:54:44.530542 32153 raft_consensus.cc:2454] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90ed6faf3425442ab2a6249ee33a62c8 in term 0.
I20260810 00:54:44.530547 32566 raft_consensus.cc:2454] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90ed6faf3425442ab2a6249ee33a62c8 in term 0.
I20260810 00:54:44.531143 32258 leader_election.cc:302] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90ed6faf3425442ab2a6249ee33a62c8, d833f3e4ccaa4e329cfec01b259b12b5; no voters:
I20260810 00:54:44.531478 32672 raft_consensus.cc:2781] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:54:44.531615 32672 raft_consensus.cc:482] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:54:44.531723 32672 raft_consensus.cc:3037] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:54:44.534025 32672 raft_consensus.cc:504] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } }
I20260810 00:54:44.534521 32672 leader_election.cc:288] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [CANDIDATE]: Term 1 election: Requested vote from peers 101a6277a18a47aaa8e2a0c52a7cd1da (127.10.97.65:42033), d833f3e4ccaa4e329cfec01b259b12b5 (127.10.97.67:34171)
I20260810 00:54:44.534905 32153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dba3e4ca651433b84289229da86927d" candidate_uuid: "90ed6faf3425442ab2a6249ee33a62c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da"
I20260810 00:54:44.534976 32566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dba3e4ca651433b84289229da86927d" candidate_uuid: "90ed6faf3425442ab2a6249ee33a62c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d833f3e4ccaa4e329cfec01b259b12b5"
I20260810 00:54:44.535061 32153 raft_consensus.cc:3037] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:54:44.535157 32566 raft_consensus.cc:3037] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:54:44.537283 32153 raft_consensus.cc:2454] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90ed6faf3425442ab2a6249ee33a62c8 in term 1.
I20260810 00:54:44.537341 32566 raft_consensus.cc:2454] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90ed6faf3425442ab2a6249ee33a62c8 in term 1.
I20260810 00:54:44.537595 32258 leader_election.cc:302] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101a6277a18a47aaa8e2a0c52a7cd1da, 90ed6faf3425442ab2a6249ee33a62c8; no voters:
I20260810 00:54:44.537825 32672 raft_consensus.cc:2781] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:54:44.538215 32672 raft_consensus.cc:686] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [term 1 LEADER]: Becoming Leader. State: Replica: 90ed6faf3425442ab2a6249ee33a62c8, State: Running, Role: LEADER
I20260810 00:54:44.538645 32672 consensus_queue.cc:237] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } }
I20260810 00:54:44.543004 31935 catalog_manager.cc:5495] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90ed6faf3425442ab2a6249ee33a62c8 (127.10.97.66). New cstate: current_term: 1 leader_uuid: "90ed6faf3425442ab2a6249ee33a62c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } health_report { overall_health: HEALTHY } } }
W20260810 00:54:44.566992 32641 tablet.cc:2369] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:54:44.581550 32657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.576944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52796 (local address 127.10.97.66:39415)
0810 00:54:44.577116 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:44.577121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:44.577137 (+ 16us) server_negotiation.cc:407] Connection header received
0810 00:54:44.577276 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:44.577280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:44.577351 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:44.577464 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:54:44.577782 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.578451 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.579710 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.580118 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.580201 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:44.580625 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:54:44.581093 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:54:44.581194 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:44.581232 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 00:54:44.581290 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 00:54:44.589651 32566 raft_consensus.cc:1261] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5 [term 1 FOLLOWER]: Refusing update from remote peer 90ed6faf3425442ab2a6249ee33a62c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:54:44.589644 32153 raft_consensus.cc:1261] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Refusing update from remote peer 90ed6faf3425442ab2a6249ee33a62c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260810 00:54:44.590546 32436 tablet.cc:2369] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:54:44.590612 32676 consensus_queue.cc:1035] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:54:44.590991 32672 consensus_queue.cc:1035] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 00:54:44.594012 32227 tablet.cc:2369] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:54:44.601107 32683 mvcc.cc:205] Tried to move back new op lower bound from 7316780173669199872 to 7316780173473824768. Current Snapshot: MvccSnapshot[applied={T|T < 7316780173669199872}]
I20260810 00:54:44.604343 32685 mvcc.cc:205] Tried to move back new op lower bound from 7316780173669199872 to 7316780173473824768. Current Snapshot: MvccSnapshot[applied={T|T < 7316780173669199872}]
I20260810 00:54:44.614192 31935 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:54:44.614480 31935 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:54:44.619481 31935 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db: Sending DeleteTablet for 3 replicas of tablet 1dba3e4ca651433b84289229da86927d
I20260810 00:54:44.620889 32331 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dba3e4ca651433b84289229da86927d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:54:44 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:52782
I20260810 00:54:44.621263 32124 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dba3e4ca651433b84289229da86927d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:54:44 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:36308
I20260810 00:54:44.621688 32538 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dba3e4ca651433b84289229da86927d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:54:44 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:42826
I20260810 00:54:44.621681 32537 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dba3e4ca651433b84289229da86927d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:54:44 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:42826
I20260810 00:54:44.622748 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 32027
I20260810 00:54:44.637189 32695 tablet_replica.cc:330] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8: stopping tablet replica
I20260810 00:54:44.637763 32700 tablet_replica.cc:330] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5: stopping tablet replica
I20260810 00:54:44.638198 32700 raft_consensus.cc:2229] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:54:44.638763 32700 raft_consensus.cc:2258] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:54:44.639319 32697 tablet_replica.cc:330] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da: stopping tablet replica
I20260810 00:54:44.639545 32695 raft_consensus.cc:2229] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:54:44.639678 32697 raft_consensus.cc:2229] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:54:44.640151 32697 raft_consensus.cc:2258] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:54:44.640197 32695 raft_consensus.cc:2258] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:54:44.642642 32695 ts_tablet_manager.cc:1905] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:54:44.642704 32697 ts_tablet_manager.cc:1905] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:54:44.644531 32700 ts_tablet_manager.cc:1905] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 00:54:44 dist-test-slave-jlzn krb5kdc[31862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:54:44.650135 32695 ts_tablet_manager.cc:1918] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:54:44.650343 32695 log.cc:1196] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/1dba3e4ca651433b84289229da86927d
I20260810 00:54:44.650964 32695 ts_tablet_manager.cc:1939] T 1dba3e4ca651433b84289229da86927d P 90ed6faf3425442ab2a6249ee33a62c8: Deleting consensus metadata
I20260810 00:54:44.652798 31922 catalog_manager.cc:4841] TS 90ed6faf3425442ab2a6249ee33a62c8 (127.10.97.66:39415): tablet 1dba3e4ca651433b84289229da86927d (table test-table [id=39ed9b7a96554e1baa672fcac97b0bd3]) successfully deleted
I20260810 00:54:44.655923 32700 ts_tablet_manager.cc:1918] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:54:44.656128 32700 log.cc:1196] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/1dba3e4ca651433b84289229da86927d
I20260810 00:54:44.656673 32700 ts_tablet_manager.cc:1939] T 1dba3e4ca651433b84289229da86927d P d833f3e4ccaa4e329cfec01b259b12b5: Deleting consensus metadata
I20260810 00:54:44.658260 32654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.623887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36320 (local address 127.10.97.65:42033)
0810 00:54:44.624050 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:44.624056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:44.632697 (+ 8641us) server_negotiation.cc:407] Connection header received
0810 00:54:44.633658 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:44.633664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:44.633757 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:44.633827 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:44.640551 (+ 6724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.641675 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.643594 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.644219 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.644317 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:44.649836 (+ 5519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:44.649849 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:44.649866 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:44.649902 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:44.653507 (+ 3605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:44.654366 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:44.654375 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:44.654379 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:44.654543 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:44.655050 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:44.655056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:44.655060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:44.655561 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:44.655933 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:44.657663 (+ 1730us) server_negotiation.cc:299] Negotiation successful
0810 00:54:44.657900 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 00:54:44.658609 32697 ts_tablet_manager.cc:1918] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:54:44.658764 32697 log.cc:1196] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/1dba3e4ca651433b84289229da86927d
I20260810 00:54:44.659322 32697 ts_tablet_manager.cc:1939] T 1dba3e4ca651433b84289229da86927d P 101a6277a18a47aaa8e2a0c52a7cd1da: Deleting consensus metadata
I20260810 00:54:44.659652 32218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:44.661129 31921 catalog_manager.cc:4841] TS d833f3e4ccaa4e329cfec01b259b12b5 (127.10.97.67:34171): tablet 1dba3e4ca651433b84289229da86927d (table test-table [id=39ed9b7a96554e1baa672fcac97b0bd3]) successfully deleted
W20260810 00:54:44.662184 31921 catalog_manager.cc:4808] TS d833f3e4ccaa4e329cfec01b259b12b5 (127.10.97.67:34171): delete failed for tablet 1dba3e4ca651433b84289229da86927d because the tablet was not found. No further retry: Not found: Tablet not found: 1dba3e4ca651433b84289229da86927d
I20260810 00:54:44.784885 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 32027
W20260810 00:54:44.814448 31921 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.97.65:42033 (error 108)
I20260810 00:54:44.814771 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 32235
I20260810 00:54:44.816713 32656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.816270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.0.0.1:36326)
0810 00:54:44.816430 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:44.816548 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:42033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 00:54:44.817883 31921 catalog_manager.cc:4584] TS 101a6277a18a47aaa8e2a0c52a7cd1da (127.10.97.65:42033): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1dba3e4ca651433b84289229da86927d: Network error: Client connection negotiation failed: client connection to 127.10.97.65:42033: connect: Connection refused (error 111)
Aug 10 00:54:44 dist-test-slave-jlzn krb5kdc[31862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.66@KRBTEST.COM
I20260810 00:54:44.832468 32657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.815987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52810 (local address 127.10.97.66:39415)
0810 00:54:44.816175 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:44.816181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:44.816201 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:54:44.816272 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:44.816277 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:44.816358 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:44.816462 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:44.817815 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.819069 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.820469 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.821156 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.821298 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:44.825439 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:44.825455 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:44.825472 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:44.825528 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:44.829371 (+ 3843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:44.830150 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:44.830159 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:44.830162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:44.830355 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:44.830735 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:44.830740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:44.830743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:44.831229 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:44.831568 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:44.831855 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 00:54:44.832087 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 00:54:44.833117 32427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:44.871527 32656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.870973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.0.0.1:36328)
0810 00:54:44.871195 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:44.871313 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:42033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 00:54:44.872589 32656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.872198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.0.0.1:36336)
0810 00:54:44.872350 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:44.872451 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:42033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 00:54:44.932911 32656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.932423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.0.0.1:36340)
0810 00:54:44.932632 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:44.932733 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:42033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 00:54:44.933902 32656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.933532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.0.0.1:36350)
0810 00:54:44.933679 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:44.933762 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:42033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 00:54:44.946988 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 32235
I20260810 00:54:44.972457 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 32442
Aug 10 00:54:44 dist-test-slave-jlzn krb5kdc[31862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.67@KRBTEST.COM
I20260810 00:54:44.986290 32660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:44.973170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42828 (local address 127.10.97.67:34171)
0810 00:54:44.973335 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:44.973340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:44.973362 (+ 22us) server_negotiation.cc:407] Connection header received
0810 00:54:44.973542 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:44.973546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:44.973611 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:44.973695 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:44.974910 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.975634 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.976932 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:44.977307 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:44.977398 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:44.981505 (+ 4107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:44.981521 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:44.981538 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:44.981570 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:44.984184 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:44.984754 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:44.984761 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:44.984763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:44.984894 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:44.985169 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:44.985171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:44.985173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:44.985500 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:44.985718 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:44.985898 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 00:54:44.986052 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 00:54:44.986893 32631 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:45.014175 32656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:45.013660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.0.0.1:36366)
0810 00:54:45.013874 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:45.013975 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:42033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 00:54:45.015169 32656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:45.014747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.0.0.1:36372)
0810 00:54:45.014932 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:45.015023 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:42033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 00:54:45.072894 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 32442
I20260810 00:54:45.096063 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 31886
I20260810 00:54:45.097151 31979 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:45.203795 32656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:45.203266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.0.0.1:36376)
0810 00:54:45.203475 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:45.203568 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:42033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 00:54:45.204232 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 31886
I20260810 00:54:45.232959 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:43387
--webserver_interface=127.10.97.126
--webserver_port=36949
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:43387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:54:45.553759 32710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:45.554024 32710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:45.554154 32710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:45.560987 32710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:54:45.561084 32710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:45.561157 32710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:45.561226 32710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:54:45.561285 32710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:54:45.570415 32710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:43387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:43387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=36949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:45.571543 32710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:45.572779 32710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:45.597184 32716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:45.597142 32717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:45.597198 32721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:45.597988 32710 server_base.cc:1034] running on GCE node
I20260810 00:54:45.598773 32710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:45.600226 32710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:45.601426 32710 hybrid_clock.cc:648] HybridClock initialized: now 1786323285601370 us; error 41 us; skew 500 ppm
I20260810 00:54:45.601799 32710 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:45 dist-test-slave-jlzn krb5kdc[31862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323285, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:45.605150 32710 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:54:45.606600 32710 webserver.cc:460] Webserver started at http://127.10.97.126:36949/ using document root <none> and password file <none>
I20260810 00:54:45.607291 32710 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:45.607427 32710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:45.612901 32710 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 00:54:45.615619 32733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:45.616863 32710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 00:54:45.617031 32710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "0cd8f87eb6cc40dd877cda931a8498db"
format_stamp: "Formatted at 2026-08-10 00:54:42 on dist-test-slave-jlzn"
server_key: "9c1d78c30d6bdd8284defd7afb2af23d"
server_key_iv: "1af3faefcd158fa099a3107e7639725f"
server_key_version: "encryptionkey@0"
I20260810 00:54:45.618037 32710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:45.638710 32710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:45.642414 32710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:45.642656 32710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:45.660739 32710 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:43387
I20260810 00:54:45.660759 337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:43387 every 8 connection(s)
I20260810 00:54:45.662098 32710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:54:45.664942 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 32710
I20260810 00:54:45.665742 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:42033
--local_ip_for_outbound_sockets=127.10.97.65
--tserver_master_addrs=127.10.97.126:43387
--webserver_port=43789
--webserver_interface=127.10.97.65
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:54:45.668932 338 sys_catalog.cc:263] Verifying existing consensus state
I20260810 00:54:45.672111 338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db: Bootstrap starting.
I20260810 00:54:45.690682 338 log.cc:824] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db: Log is configured to *not* fsync() on all Append() calls
I20260810 00:54:45.702416 338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 00:54:45.702913 338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db: Bootstrap complete.
I20260810 00:54:45.711655 338 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd8f87eb6cc40dd877cda931a8498db" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43387 } }
I20260810 00:54:45.712508 338 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd8f87eb6cc40dd877cda931a8498db, State: Initialized, Role: FOLLOWER
I20260810 00:54:45.713060 338 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd8f87eb6cc40dd877cda931a8498db" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43387 } }
I20260810 00:54:45.713289 338 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:54:45.713402 338 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:54:45.713558 338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:54:45.717164 338 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd8f87eb6cc40dd877cda931a8498db" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43387 } }
I20260810 00:54:45.717636 338 leader_election.cc:302] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cd8f87eb6cc40dd877cda931a8498db; no voters:
I20260810 00:54:45.718201 338 leader_election.cc:288] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 00:54:45.718333 343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 2 FOLLOWER]: Leader election won for term 2
I20260810 00:54:45.719446 343 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 2 LEADER]: Becoming Leader. State: Replica: 0cd8f87eb6cc40dd877cda931a8498db, State: Running, Role: LEADER
I20260810 00:54:45.720053 343 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd8f87eb6cc40dd877cda931a8498db" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43387 } }
I20260810 00:54:45.720345 338 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:54:45.728691 345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cd8f87eb6cc40dd877cda931a8498db. Latest consensus state: current_term: 2 leader_uuid: "0cd8f87eb6cc40dd877cda931a8498db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd8f87eb6cc40dd877cda931a8498db" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43387 } } }
I20260810 00:54:45.729055 345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [sys.catalog]: This master's current role is: LEADER
I20260810 00:54:45.738125 344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0cd8f87eb6cc40dd877cda931a8498db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd8f87eb6cc40dd877cda931a8498db" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43387 } } }
I20260810 00:54:45.738451 344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [sys.catalog]: This master's current role is: LEADER
I20260810 00:54:45.738771 361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:54:45.746654 361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:54:45.748171 361 catalog_manager.cc:1259] Loaded cluster ID: fa63f84ed5a1400686fb322b339b7056
I20260810 00:54:45.748312 361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:54:45.751576 361 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 00:54:45.769229 361 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 00:54:45.778101 361 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:54:45.779700 361 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db: Loaded TSK: 0
I20260810 00:54:45.781177 361 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 00:54:46.002525 340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:46.002797 340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:46.002930 340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:46.009966 340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:46.010076 340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:46.010215 340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:54:46.019409 340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:42033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=43789
--tserver_master_addrs=127.10.97.126:43387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:46.020684 340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:46.022096 340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:46.035995 384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:46.036031 385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:46.035953 388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:46.036942 340 server_base.cc:1034] running on GCE node
I20260810 00:54:46.037508 340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:46.038508 340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:46.039714 340 hybrid_clock.cc:648] HybridClock initialized: now 1786323286039657 us; error 46 us; skew 500 ppm
I20260810 00:54:46.040048 340 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:46 dist-test-slave-jlzn krb5kdc[31862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323286, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:46.046288 340 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:54:46.047739 340 webserver.cc:460] Webserver started at http://127.10.97.65:43789/ using document root <none> and password file <none>
I20260810 00:54:46.048439 340 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:46.048574 340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:46.053751 340 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260810 00:54:46.056577 401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:46.057874 340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:54:46.058024 340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "101a6277a18a47aaa8e2a0c52a7cd1da"
format_stamp: "Formatted at 2026-08-10 00:54:43 on dist-test-slave-jlzn"
server_key: "3afb52af1450e397d5818f94648093de"
server_key_iv: "fc629ac818c3b9b356144d02f518ef96"
server_key_version: "encryptionkey@0"
I20260810 00:54:46.058929 340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:46.075062 340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:46.078835 340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:46.079064 340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:46.079993 340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:54:46.081255 340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:54:46.081357 340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:46.081461 340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:54:46.081532 340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:46.110244 340 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:42033
I20260810 00:54:46.110307 585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:42033 every 8 connection(s)
I20260810 00:54:46.111666 340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:54:46.121362 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 340
I20260810 00:54:46.122098 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:39415
--local_ip_for_outbound_sockets=127.10.97.66
--tserver_master_addrs=127.10.97.126:43387
--webserver_port=35189
--webserver_interface=127.10.97.66
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 00:54:46 dist-test-slave-jlzn krb5kdc[31862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323286, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:54:46.134544 589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:46.114289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:53579 (local address 127.10.97.126:43387)
0810 00:54:46.115011 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:46.115026 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:46.115439 (+ 413us) server_negotiation.cc:407] Connection header received
0810 00:54:46.117399 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:46.117411 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:46.117711 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:46.118016 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:46.119486 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:46.120755 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:46.122982 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:46.123593 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:46.123728 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:46.127366 (+ 3638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:46.127376 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:46.127394 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:46.127444 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:46.130699 (+ 3255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:46.131417 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:46.131425 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:46.131432 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:46.131646 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:46.132046 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:46.132050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:46.132053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:46.132623 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:46.132802 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:46.133413 (+ 611us) server_negotiation.cc:299] Negotiation successful
0810 00:54:46.133704 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":340,"threads_started":1}
I20260810 00:54:46.134853 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:46.114504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:53579)
0810 00:54:46.115282 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:46.115327 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:46.116927 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:46.118196 (+ 1269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:46.118208 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:46.118754 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:46.119267 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:46.119285 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:46.120979 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:46.120986 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:46.122846 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:46.122861 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:46.123846 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:46.123854 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:46.124137 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:46.124758 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:46.124784 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:46.127103 (+ 2319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:46.130863 (+ 3760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:46.130872 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:46.130884 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:46.131292 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:46.131746 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:46.131749 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:46.131751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:46.131953 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:46.132730 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:46.132735 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:46.132992 (+ 257us) client_negotiation.cc:769] Sending connection context
0810 00:54:46.133239 (+ 247us) client_negotiation.cc:240] Negotiation successful
0810 00:54:46.133703 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":309,"threads_started":1}
I20260810 00:54:46.137722 586 heartbeater.cc:347] Connected to a master server at 127.10.97.126:43387
I20260810 00:54:46.137995 586 heartbeater.cc:464] Registering TS with master...
I20260810 00:54:46.138729 586 heartbeater.cc:511] Master 127.10.97.126:43387 requested a full tablet report, sending...
I20260810 00:54:46.140903 32756 ts_manager.cc:194] Registered new tserver with Master: 101a6277a18a47aaa8e2a0c52a7cd1da (127.10.97.65:42033)
I20260810 00:54:46.142817 32756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:53579
W20260810 00:54:46.439885 592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:46.440128 592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:46.440235 592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:46.447045 592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:46.447147 592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:46.447259 592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:54:46.456265 592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:39415
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=35189
--tserver_master_addrs=127.10.97.126:43387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:46.457288 592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:46.458464 592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:46.470546 604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:46.470636 597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:46.470762 600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:46.471539 592 server_base.cc:1034] running on GCE node
I20260810 00:54:46.472102 592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:46.472877 592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:46.474066 592 hybrid_clock.cc:648] HybridClock initialized: now 1786323286474028 us; error 28 us; skew 500 ppm
I20260810 00:54:46.474355 592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:46 dist-test-slave-jlzn krb5kdc[31862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323286, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:46.477205 592 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:54:46.478458 592 webserver.cc:460] Webserver started at http://127.10.97.66:35189/ using document root <none> and password file <none>
I20260810 00:54:46.479044 592 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:46.479176 592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:46.484202 592 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 00:54:46.487123 614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:46.488358 592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:54:46.488495 592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "90ed6faf3425442ab2a6249ee33a62c8"
format_stamp: "Formatted at 2026-08-10 00:54:43 on dist-test-slave-jlzn"
server_key: "e080b34aa8b3334c315c0756d892a049"
server_key_iv: "0c529fcc224024c89a889c99edef76f8"
server_key_version: "encryptionkey@0"
I20260810 00:54:46.489413 592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:46.509240 592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:46.512776 592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:46.512985 592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:46.513818 592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:54:46.515029 592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:54:46.515111 592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:46.515201 592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:54:46.515259 592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:46.544057 592 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:39415
I20260810 00:54:46.544122 773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:39415 every 8 connection(s)
I20260810 00:54:46.545425 592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:54:46.550930 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 592
I20260810 00:54:46.551525 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:34171
--local_ip_for_outbound_sockets=127.10.97.67
--tserver_master_addrs=127.10.97.126:43387
--webserver_port=46697
--webserver_interface=127.10.97.67
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 00:54:46 dist-test-slave-jlzn krb5kdc[31862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323286, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:54:46.563827 589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:46.547761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:60733 (local address 127.10.97.126:43387)
0810 00:54:46.547972 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:46.547977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:46.549049 (+ 1072us) server_negotiation.cc:407] Connection header received
0810 00:54:46.550813 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:46.550820 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:46.550899 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:46.550989 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:46.552508 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:46.553198 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:46.554692 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:46.555112 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:46.555230 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:46.558588 (+ 3358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:46.558593 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:46.558597 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:46.558636 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:46.560906 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:46.561569 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:46.561573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:46.561575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:46.561706 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:46.562149 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:46.562154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:46.562157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:46.562428 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:46.562561 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:46.563220 (+ 659us) server_negotiation.cc:299] Negotiation successful
0810 00:54:46.563455 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 00:54:46.564649 777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:46.548229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:60733)
0810 00:54:46.548892 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:46.548935 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:46.550544 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:46.551241 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:46.551248 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:46.551787 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:46.552346 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:46.552362 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:46.553351 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:46.553355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:46.554547 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:46.554558 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:46.555333 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:46.555341 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:46.555542 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:46.556242 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:46.556269 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:46.558399 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:46.561030 (+ 2631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:46.561039 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:46.561053 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:46.561447 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:46.561808 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:46.561813 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:46.561817 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:46.562017 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:46.562551 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:46.562559 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:46.562847 (+ 288us) client_negotiation.cc:769] Sending connection context
0810 00:54:46.563112 (+ 265us) client_negotiation.cc:240] Negotiation successful
0810 00:54:46.563605 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":250,"threads_started":1}
I20260810 00:54:46.565973 774 heartbeater.cc:347] Connected to a master server at 127.10.97.126:43387
I20260810 00:54:46.566272 774 heartbeater.cc:464] Registering TS with master...
I20260810 00:54:46.567046 774 heartbeater.cc:511] Master 127.10.97.126:43387 requested a full tablet report, sending...
I20260810 00:54:46.568637 32756 ts_manager.cc:194] Registered new tserver with Master: 90ed6faf3425442ab2a6249ee33a62c8 (127.10.97.66:39415)
I20260810 00:54:46.569687 32756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:60733
W20260810 00:54:46.875633 778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:46.875908 778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:46.876014 778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:46.882946 778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:46.883040 778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:46.883152 778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:54:46.892309 778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:34171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=46697
--tserver_master_addrs=127.10.97.126:43387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:46.893473 778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:46.894707 778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:46.906869 790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:46.906911 785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:46.906998 788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:46.908092 778 server_base.cc:1034] running on GCE node
I20260810 00:54:46.908687 778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:46.909540 778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:46.910758 778 hybrid_clock.cc:648] HybridClock initialized: now 1786323286910709 us; error 34 us; skew 500 ppm
I20260810 00:54:46.911064 778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:46 dist-test-slave-jlzn krb5kdc[31862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323286, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:46.913975 778 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:54:46.915347 778 webserver.cc:460] Webserver started at http://127.10.97.67:46697/ using document root <none> and password file <none>
I20260810 00:54:46.916038 778 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:46.916173 778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:46.921126 778 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 00:54:46.923702 802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:46.924918 778 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:54:46.925073 778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "d833f3e4ccaa4e329cfec01b259b12b5"
format_stamp: "Formatted at 2026-08-10 00:54:44 on dist-test-slave-jlzn"
server_key: "4375b6f8ffe702189e57a4a0feb428a2"
server_key_iv: "63cf1f9f1cbe16c4534ec62f3a7061d1"
server_key_version: "encryptionkey@0"
I20260810 00:54:46.925998 778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:46.949973 778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:46.953444 778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:46.953687 778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:46.954559 778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:54:46.955772 778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:54:46.955892 778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:46.955996 778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:54:46.956068 778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:46.985646 778 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:34171
I20260810 00:54:46.985721 969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:34171 every 8 connection(s)
I20260810 00:54:46.986980 778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:54:46.989547 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 778
Aug 10 00:54:46 dist-test-slave-jlzn krb5kdc[31862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323286, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:54:47.004580 975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:46.990939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38936 (local address 127.10.97.126:43387)
0810 00:54:46.991385 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:46.991389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:46.991402 (+ 13us) server_negotiation.cc:407] Connection header received
0810 00:54:46.991454 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:46.991458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:46.991530 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:46.991624 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:46.993226 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:46.994051 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:46.995401 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:46.995789 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:46.995924 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:46.998718 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:46.998723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:46.998726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:46.998757 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:47.001966 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:47.002588 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:47.002598 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:47.002602 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:47.002739 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:47.003101 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:47.003107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:47.003110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:47.003385 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:47.003627 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:47.003971 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 00:54:47.004183 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":243,"threads_started":1}
I20260810 00:54:47.006381 589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:46.989339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:35825 (local address 127.10.97.126:43387)
0810 00:54:46.989542 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:46.989548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:46.990639 (+ 1091us) server_negotiation.cc:407] Connection header received
0810 00:54:46.992682 (+ 2043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:46.992688 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:46.992760 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:46.992842 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:46.994368 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:46.995072 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:46.996595 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:46.997214 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:46.997342 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:47.000379 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:47.000386 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:47.000390 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:47.000444 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:47.003181 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:47.003846 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:47.003851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:47.003855 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:47.003982 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:47.004381 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:47.004385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:47.004387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:47.004647 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:47.004777 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:47.005597 (+ 820us) server_negotiation.cc:299] Negotiation successful
0810 00:54:47.005800 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66,"spinlock_wait_cycles":3840}
I20260810 00:54:47.007085 973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:46.989724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:35825)
0810 00:54:46.990476 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:46.990519 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:46.992449 (+ 1930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:46.993062 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:46.993069 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:46.993578 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:46.994219 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:46.994239 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:46.995217 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:46.995221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:46.996450 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:46.996465 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:46.997403 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:46.997407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:46.997588 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:46.998109 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:46.998132 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:47.000167 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:47.003308 (+ 3141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:47.003315 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:47.003324 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:47.003704 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:47.004086 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:47.004090 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:47.004092 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:47.004267 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:47.004857 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:47.004865 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:47.005187 (+ 322us) client_negotiation.cc:769] Sending connection context
0810 00:54:47.005477 (+ 290us) client_negotiation.cc:240] Negotiation successful
0810 00:54:47.005942 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":307,"threads_started":1}
I20260810 00:54:47.008248 970 heartbeater.cc:347] Connected to a master server at 127.10.97.126:43387
I20260810 00:54:47.008520 970 heartbeater.cc:464] Registering TS with master...
I20260810 00:54:47.009239 970 heartbeater.cc:511] Master 127.10.97.126:43387 requested a full tablet report, sending...
I20260810 00:54:47.010537 32756 ts_manager.cc:194] Registered new tserver with Master: d833f3e4ccaa4e329cfec01b259b12b5 (127.10.97.67:34171)
I20260810 00:54:47.011315 32756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:35825
I20260810 00:54:47.018946 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:54:47.025007 589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.019769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38948 (local address 127.10.97.126:43387)
0810 00:54:47.019956 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:47.019960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:47.020366 (+ 406us) server_negotiation.cc:407] Connection header received
0810 00:54:47.020531 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:47.020535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:47.020683 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:47.020790 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:54:47.021163 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.021876 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.023306 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.023710 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.023798 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:47.024244 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:54:47.024643 (+ 399us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:54:47.024751 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:47.024786 (+ 35us) server_negotiation.cc:299] Negotiation successful
0810 00:54:47.024842 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260810 00:54:47.025753 32756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:38948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:54:47.032635 32756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:54:47.055069 984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.043541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:39415 (local address 127.0.0.1:52826)
0810 00:54:47.046046 (+ 2505us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:47.046065 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:47.046172 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:47.047092 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:47.047097 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:47.047121 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:54:47.047527 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:47.047542 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.049553 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.049558 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:47.052315 (+ 2757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:47.052331 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.054061 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.054068 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:47.054259 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:47.054332 (+ 73us) client_negotiation.cc:769] Sending connection context
0810 00:54:47.054461 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 00:54:47.054657 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2390,"mutex_wait_us":520,"spinlock_wait_cycles":183168,"thread_start_us":1305,"threads_started":1}
I20260810 00:54:47.055955 983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.043604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52826 (local address 127.10.97.66:39415)
0810 00:54:47.045768 (+ 2164us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:47.045777 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:47.046304 (+ 527us) server_negotiation.cc:407] Connection header received
0810 00:54:47.046412 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:47.046418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:47.046617 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:47.046819 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:54:47.047773 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.049383 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.052480 (+ 3097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.053866 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.053972 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:47.054203 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:47.055457 (+ 1254us) server_negotiation.cc:299] Negotiation successful
0810 00:54:47.055606 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1990,"thread_start_us":1851,"threads_started":1}
I20260810 00:54:47.056360 985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.043062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.0.0.1:36398)
0810 00:54:47.045236 (+ 2174us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:47.045283 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:47.045524 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:47.046437 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:47.046443 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:47.046470 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:54:47.047864 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:47.047878 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.049683 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.049691 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:47.053596 (+ 3905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:47.053615 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.055584 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.055590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:47.055774 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:47.055796 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 00:54:47.055895 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 00:54:47.056011 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1931,"mutex_wait_us":41,"thread_start_us":2723,"threads_started":1}
I20260810 00:54:47.056406 982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.043097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36398 (local address 127.10.97.65:42033)
0810 00:54:47.043736 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:47.043744 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:47.045391 (+ 1647us) server_negotiation.cc:407] Connection header received
0810 00:54:47.045758 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:47.045766 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:47.045993 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:47.046161 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:54:47.048036 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.049494 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.053768 (+ 4274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.055124 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.055257 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:47.055423 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:47.055967 (+ 544us) server_negotiation.cc:299] Negotiation successful
0810 00:54:47.056130 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":370,"threads_started":1}
I20260810 00:54:47.057968 986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.046148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:34171 (local address 127.0.0.1:42840)
0810 00:54:47.050520 (+ 4372us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:47.050535 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:47.050645 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:47.051668 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:47.051672 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:47.051692 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:54:47.052156 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:47.052170 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.053505 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.053508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:47.055550 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:47.055566 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.057220 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.057227 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:47.057388 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:47.057421 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 00:54:47.057498 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 00:54:47.057622 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4238,"mutex_wait_us":3,"thread_start_us":4109,"threads_started":1}
I20260810 00:54:47.057996 989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.046424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42840 (local address 127.10.97.67:34171)
0810 00:54:47.051166 (+ 4742us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:47.051174 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:47.051205 (+ 31us) server_negotiation.cc:407] Connection header received
0810 00:54:47.051282 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:47.051289 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:47.051473 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:47.051656 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:54:47.052364 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.053378 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.055702 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.057024 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.057144 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:47.057352 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:47.057562 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 00:54:47.057707 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4598,"thread_start_us":4454,"threads_started":1}
I20260810 00:54:47.059701 682 tablet_service.cc:1467] Processing CreateTablet for tablet 099d179eee004843a050f68f9de25f0f (DEFAULT_TABLE table=test-table [id=d0835ecf32f44ab187071cb4709b6982]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:54:47.059702 484 tablet_service.cc:1467] Processing CreateTablet for tablet 099d179eee004843a050f68f9de25f0f (DEFAULT_TABLE table=test-table [id=d0835ecf32f44ab187071cb4709b6982]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:54:47.061115 484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 099d179eee004843a050f68f9de25f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:54:47.061115 682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 099d179eee004843a050f68f9de25f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:54:47.061018 867 tablet_service.cc:1467] Processing CreateTablet for tablet 099d179eee004843a050f68f9de25f0f (DEFAULT_TABLE table=test-table [id=d0835ecf32f44ab187071cb4709b6982]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:54:47.062107 867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 099d179eee004843a050f68f9de25f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:54:47.073975 990 tablet_bootstrap.cc:492] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5: Bootstrap starting.
I20260810 00:54:47.075755 994 tablet_bootstrap.cc:492] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da: Bootstrap starting.
I20260810 00:54:47.077111 993 tablet_bootstrap.cc:492] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8: Bootstrap starting.
I20260810 00:54:47.078958 990 tablet_bootstrap.cc:654] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5: Neither blocks nor log segments found. Creating new log.
I20260810 00:54:47.078958 994 tablet_bootstrap.cc:654] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da: Neither blocks nor log segments found. Creating new log.
I20260810 00:54:47.079559 993 tablet_bootstrap.cc:654] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8: Neither blocks nor log segments found. Creating new log.
I20260810 00:54:47.080230 994 log.cc:824] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da: Log is configured to *not* fsync() on all Append() calls
I20260810 00:54:47.080230 990 log.cc:824] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5: Log is configured to *not* fsync() on all Append() calls
I20260810 00:54:47.080559 993 log.cc:824] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8: Log is configured to *not* fsync() on all Append() calls
I20260810 00:54:47.083030 993 tablet_bootstrap.cc:492] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8: No bootstrap required, opened a new log
I20260810 00:54:47.083163 994 tablet_bootstrap.cc:492] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da: No bootstrap required, opened a new log
I20260810 00:54:47.083243 990 tablet_bootstrap.cc:492] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5: No bootstrap required, opened a new log
I20260810 00:54:47.083295 993 ts_tablet_manager.cc:1397] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260810 00:54:47.083422 994 ts_tablet_manager.cc:1397] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260810 00:54:47.083447 990 ts_tablet_manager.cc:1397] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260810 00:54:47.090458 993 raft_consensus.cc:348] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } }
I20260810 00:54:47.090790 993 raft_consensus.cc:374] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:54:47.090903 993 raft_consensus.cc:729] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90ed6faf3425442ab2a6249ee33a62c8, State: Initialized, Role: FOLLOWER
I20260810 00:54:47.091436 993 consensus_queue.cc:260] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } }
I20260810 00:54:47.092465 774 heartbeater.cc:503] Master 127.10.97.126:43387 was elected leader, sending a full tablet report...
I20260810 00:54:47.093148 993 ts_tablet_manager.cc:1428] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8: Time spent starting tablet: real 0.010s user 0.009s sys 0.002s
I20260810 00:54:47.093413 994 raft_consensus.cc:348] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } }
I20260810 00:54:47.093411 990 raft_consensus.cc:348] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } }
I20260810 00:54:47.093832 994 raft_consensus.cc:374] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:54:47.093832 990 raft_consensus.cc:374] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:54:47.093930 990 raft_consensus.cc:729] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d833f3e4ccaa4e329cfec01b259b12b5, State: Initialized, Role: FOLLOWER
I20260810 00:54:47.093969 994 raft_consensus.cc:729] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101a6277a18a47aaa8e2a0c52a7cd1da, State: Initialized, Role: FOLLOWER
I20260810 00:54:47.094493 990 consensus_queue.cc:260] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } }
I20260810 00:54:47.094493 994 consensus_queue.cc:260] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } }
I20260810 00:54:47.096458 586 heartbeater.cc:503] Master 127.10.97.126:43387 was elected leader, sending a full tablet report...
I20260810 00:54:47.097016 994 ts_tablet_manager.cc:1428] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da: Time spent starting tablet: real 0.013s user 0.012s sys 0.003s
I20260810 00:54:47.098065 970 heartbeater.cc:503] Master 127.10.97.126:43387 was elected leader, sending a full tablet report...
I20260810 00:54:47.098519 990 ts_tablet_manager.cc:1428] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
W20260810 00:54:47.113557 588 tablet.cc:2369] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:54:47.238446 972 tablet.cc:2369] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:54:47.297164 776 tablet.cc:2369] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:54:47.368513 1006 raft_consensus.cc:482] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:54:47.368803 1006 raft_consensus.cc:504] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } }
I20260810 00:54:47.370489 1006 leader_election.cc:288] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 101a6277a18a47aaa8e2a0c52a7cd1da (127.10.97.65:42033), 90ed6faf3425442ab2a6249ee33a62c8 (127.10.97.66:39415)
I20260810 00:54:47.377279 982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.371068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:39265 (local address 127.10.97.65:42033)
0810 00:54:47.371285 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:47.371291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:47.371315 (+ 24us) server_negotiation.cc:407] Connection header received
0810 00:54:47.371535 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:47.371541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:47.371617 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:47.371693 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:54:47.372169 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.373095 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.375330 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.376469 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.376626 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:47.376700 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:47.376900 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 00:54:47.377082 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 00:54:47.377306 973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.370833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.10.97.67:39265)
0810 00:54:47.371221 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:47.371241 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:47.371372 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:47.371774 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:47.371779 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:47.371834 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:54:47.372031 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:47.372040 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.373252 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.373256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:47.375178 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:47.375194 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.376619 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.376623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:47.376753 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:47.376779 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 00:54:47.376850 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 00:54:47.377009 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244}
I20260810 00:54:47.377715 1007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.370976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:39415 (local address 127.10.97.67:48657)
0810 00:54:47.371716 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:47.371733 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:47.371867 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:47.372161 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:47.372164 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:47.372184 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:54:47.372468 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:47.372476 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.373621 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.373627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:47.375508 (+ 1881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:47.375519 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.377031 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.377037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:47.377204 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:47.377233 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 00:54:47.377333 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 00:54:47.377450 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":390,"threads_started":1}
I20260810 00:54:47.377724 983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.371007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:48657 (local address 127.10.97.66:39415)
0810 00:54:47.371221 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:47.371227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:47.371848 (+ 621us) server_negotiation.cc:407] Connection header received
0810 00:54:47.371991 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:47.371995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:47.372051 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:47.372129 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:54:47.372616 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.373479 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.375655 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.376806 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.376965 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:47.377045 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:47.377369 (+ 324us) server_negotiation.cc:299] Negotiation successful
0810 00:54:47.377495 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 00:54:47.378345 509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099d179eee004843a050f68f9de25f0f" candidate_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" is_pre_election: true
I20260810 00:54:47.378795 714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099d179eee004843a050f68f9de25f0f" candidate_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ed6faf3425442ab2a6249ee33a62c8" is_pre_election: true
I20260810 00:54:47.378964 509 raft_consensus.cc:2454] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d833f3e4ccaa4e329cfec01b259b12b5 in term 0.
I20260810 00:54:47.379313 714 raft_consensus.cc:2454] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d833f3e4ccaa4e329cfec01b259b12b5 in term 0.
I20260810 00:54:47.379673 805 leader_election.cc:302] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101a6277a18a47aaa8e2a0c52a7cd1da, d833f3e4ccaa4e329cfec01b259b12b5; no voters:
I20260810 00:54:47.380080 1006 raft_consensus.cc:2781] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:54:47.380208 1006 raft_consensus.cc:482] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:54:47.380306 1006 raft_consensus.cc:3037] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:54:47.382414 1006 raft_consensus.cc:504] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } }
I20260810 00:54:47.382975 1006 leader_election.cc:288] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [CANDIDATE]: Term 1 election: Requested vote from peers 101a6277a18a47aaa8e2a0c52a7cd1da (127.10.97.65:42033), 90ed6faf3425442ab2a6249ee33a62c8 (127.10.97.66:39415)
I20260810 00:54:47.383369 509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099d179eee004843a050f68f9de25f0f" candidate_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da"
I20260810 00:54:47.383422 714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099d179eee004843a050f68f9de25f0f" candidate_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ed6faf3425442ab2a6249ee33a62c8"
I20260810 00:54:47.383581 509 raft_consensus.cc:3037] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:54:47.383582 714 raft_consensus.cc:3037] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:54:47.386667 714 raft_consensus.cc:2454] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d833f3e4ccaa4e329cfec01b259b12b5 in term 1.
I20260810 00:54:47.386659 509 raft_consensus.cc:2454] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d833f3e4ccaa4e329cfec01b259b12b5 in term 1.
I20260810 00:54:47.387045 806 leader_election.cc:302] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90ed6faf3425442ab2a6249ee33a62c8, d833f3e4ccaa4e329cfec01b259b12b5; no voters:
I20260810 00:54:47.387375 1006 raft_consensus.cc:2781] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:54:47.387790 1006 raft_consensus.cc:686] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [term 1 LEADER]: Becoming Leader. State: Replica: d833f3e4ccaa4e329cfec01b259b12b5, State: Running, Role: LEADER
I20260810 00:54:47.388278 1006 consensus_queue.cc:237] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } }
I20260810 00:54:47.392557 32756 catalog_manager.cc:5495] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to d833f3e4ccaa4e329cfec01b259b12b5 (127.10.97.67). New cstate: current_term: 1 leader_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } health_report { overall_health: HEALTHY } } }
I20260810 00:54:47.418110 989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.413405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42848 (local address 127.10.97.67:34171)
0810 00:54:47.413593 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:47.413598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:47.413682 (+ 84us) server_negotiation.cc:407] Connection header received
0810 00:54:47.413845 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:47.413848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:47.413926 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:47.414037 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:54:47.414308 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.414986 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.416337 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.416724 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.416811 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:47.417207 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:54:47.417673 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:54:47.417770 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:47.417823 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 00:54:47.417881 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 00:54:47.425734 714 raft_consensus.cc:1261] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8 [term 1 FOLLOWER]: Refusing update from remote peer d833f3e4ccaa4e329cfec01b259b12b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:54:47.425734 509 raft_consensus.cc:1261] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Refusing update from remote peer d833f3e4ccaa4e329cfec01b259b12b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:54:47.426685 1008 consensus_queue.cc:1035] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:54:47.427031 1006 consensus_queue.cc:1035] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:54:47.438540 1015 mvcc.cc:205] Tried to move back new op lower bound from 7316780185286615040 to 7316780185145921536. Current Snapshot: MvccSnapshot[applied={T|T < 7316780185286615040}]
I20260810 00:54:47.439922 1017 mvcc.cc:205] Tried to move back new op lower bound from 7316780185286615040 to 7316780185145921536. Current Snapshot: MvccSnapshot[applied={T|T < 7316780185286615040}]
I20260810 00:54:47.454516 32754 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:38948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:54:47.454788 32754 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:38948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:54:47.458222 32754 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db: Sending DeleteTablet for 3 replicas of tablet 099d179eee004843a050f68f9de25f0f
I20260810 00:54:47.459264 682 tablet_service.cc:1514] Processing DeleteTablet for tablet 099d179eee004843a050f68f9de25f0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:54:47 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:52826
I20260810 00:54:47.459340 484 tablet_service.cc:1514] Processing DeleteTablet for tablet 099d179eee004843a050f68f9de25f0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:54:47 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:36398
I20260810 00:54:47.459581 867 tablet_service.cc:1514] Processing DeleteTablet for tablet 099d179eee004843a050f68f9de25f0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:54:47 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:42840
I20260810 00:54:47.459767 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 340
I20260810 00:54:47.460961 1033 tablet_replica.cc:330] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5: stopping tablet replica
I20260810 00:54:47.462426 1033 raft_consensus.cc:2229] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:54:47.463090 1033 raft_consensus.cc:2258] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:54:47.474368 1033 ts_tablet_manager.cc:1905] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:54:47.477869 982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.460547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36404 (local address 127.10.97.65:42033)
0810 00:54:47.460723 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:47.460729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:47.460856 (+ 127us) server_negotiation.cc:407] Connection header received
0810 00:54:47.461033 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:47.461039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:47.461123 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:47.461228 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:47.463014 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.464195 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.466246 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.466907 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.467025 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:47.470676 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:47.470689 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:47.470705 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:47.470744 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:47.474441 (+ 3697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:47.475230 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:47.475239 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:47.475242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:47.475402 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:47.476183 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:47.476188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:47.476192 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:47.476658 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:47.476953 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:47.477267 (+ 314us) server_negotiation.cc:299] Negotiation successful
0810 00:54:47.477490 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 00:54:47.478673 579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:47.484671 1033 ts_tablet_manager.cc:1918] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:54:47.484923 1033 log.cc:1196] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/099d179eee004843a050f68f9de25f0f
I20260810 00:54:47.485610 1033 ts_tablet_manager.cc:1939] T 099d179eee004843a050f68f9de25f0f P d833f3e4ccaa4e329cfec01b259b12b5: Deleting consensus metadata
I20260810 00:54:47.487574 32738 catalog_manager.cc:4841] TS d833f3e4ccaa4e329cfec01b259b12b5 (127.10.97.67:34171): tablet 099d179eee004843a050f68f9de25f0f (table test-table [id=d0835ecf32f44ab187071cb4709b6982]) successfully deleted
I20260810 00:54:47.493388 1031 tablet_replica.cc:330] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8: stopping tablet replica
I20260810 00:54:47.493957 1031 raft_consensus.cc:2229] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:54:47.494678 1031 raft_consensus.cc:2258] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:54:47.496515 1031 ts_tablet_manager.cc:1905] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:54:47.501458 1031 ts_tablet_manager.cc:1918] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:54:47.501609 1031 log.cc:1196] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/099d179eee004843a050f68f9de25f0f
I20260810 00:54:47.502148 1031 ts_tablet_manager.cc:1939] T 099d179eee004843a050f68f9de25f0f P 90ed6faf3425442ab2a6249ee33a62c8: Deleting consensus metadata
I20260810 00:54:47.503751 32739 catalog_manager.cc:4841] TS 90ed6faf3425442ab2a6249ee33a62c8 (127.10.97.66:39415): tablet 099d179eee004843a050f68f9de25f0f (table test-table [id=d0835ecf32f44ab187071cb4709b6982]) successfully deleted
I20260810 00:54:47.584975 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 340
I20260810 00:54:47.584945 1032 tablet_replica.cc:330] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da: stopping tablet replica
W20260810 00:54:47.614472 32738 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.97.65:42033 (error 108)
I20260810 00:54:47.614821 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 592
I20260810 00:54:47.617277 1037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.616084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.0.0.1:36406)
0810 00:54:47.616922 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:47.617059 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:42033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":600,"threads_started":1}
W20260810 00:54:47.617550 32738 catalog_manager.cc:4584] TS 101a6277a18a47aaa8e2a0c52a7cd1da (127.10.97.65:42033): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 099d179eee004843a050f68f9de25f0f: Network error: Client connection negotiation failed: client connection to 127.10.97.65:42033: connect: Connection refused (error 111)
I20260810 00:54:47.628890 983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.615961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52832 (local address 127.10.97.66:39415)
0810 00:54:47.616128 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:47.616133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:47.616206 (+ 73us) server_negotiation.cc:407] Connection header received
0810 00:54:47.616402 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:47.616407 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:47.616480 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:47.616563 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:47.618309 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.619015 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.620438 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.620851 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.620980 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:47.623491 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:47.623503 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:47.623517 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:47.623550 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:47.626249 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:47.626907 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:47.626913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:47.626916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:47.627041 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:47.627400 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:47.627404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:47.627406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:47.627757 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:47.628063 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:47.628335 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 00:54:47.628542 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 00:54:47.629508 765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:47.671247 1037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.670734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.0.0.1:36416)
0810 00:54:47.670975 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:47.671085 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:42033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260810 00:54:47.674991 1037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.672061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.0.0.1:36430)
0810 00:54:47.674744 (+ 2683us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:47.674826 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:42033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2590,"threads_started":1}
I20260810 00:54:47.722174 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 592
I20260810 00:54:47.741879 1040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.741399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.0.0.1:36446)
0810 00:54:47.741597 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:47.741708 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:42033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 00:54:47.742908 1040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.742556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.0.0.1:36448)
0810 00:54:47.742701 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:47.742789 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:42033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 00:54:47.753751 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 778
I20260810 00:54:47.769593 989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.754565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42854 (local address 127.10.97.67:34171)
0810 00:54:47.754724 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:47.754729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:47.754774 (+ 45us) server_negotiation.cc:407] Connection header received
0810 00:54:47.754960 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:47.754964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:47.755032 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:47.755115 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:47.757052 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.757869 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.759748 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:47.760305 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:47.760394 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:47.763660 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:47.763670 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:47.763688 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:47.763723 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:47.766675 (+ 2952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:47.767486 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:47.767492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:47.767494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:47.767641 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:47.768084 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:47.768091 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:47.768093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:47.768437 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:47.768769 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:47.769060 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 00:54:47.769230 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 00:54:47.770275 960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:47.823354 1040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.822806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.0.0.1:36458)
0810 00:54:47.823027 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:47.823137 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:42033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 00:54:47.824412 1040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:47.823982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.0.0.1:36464)
0810 00:54:47.824179 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:47.824264 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:42033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260810 00:54:47.889339 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 778
I20260810 00:54:47.913420 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 32710
I20260810 00:54:47.914510 332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:54:48.024348 1040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:48.023744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:42033 (local address 127.0.0.1:36472)
0810 00:54:48.024006 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:48.024113 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:42033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260810 00:54:48.024633 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 32710
I20260810 00:54:48.049721 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:43387
--webserver_interface=127.10.97.126
--webserver_port=36949
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:43387
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:54:48.368469 1044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:48.368731 1044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:48.368860 1044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:48.375713 1044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:54:48.375826 1044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:48.375906 1044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:48.375981 1044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:54:48.376037 1044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:54:48.385128 1044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:43387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:43387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=36949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:48.386233 1044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:48.387437 1044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:48.397943 1054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:48.398039 1056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:48.398017 1061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:48.398970 1044 server_base.cc:1034] running on GCE node
I20260810 00:54:48.399715 1044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:48.400946 1044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:48.402143 1044 hybrid_clock.cc:648] HybridClock initialized: now 1786323288402092 us; error 40 us; skew 500 ppm
I20260810 00:54:48.402446 1044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:48 dist-test-slave-jlzn krb5kdc[31862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323288, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:48.405308 1044 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:54:48.406643 1044 webserver.cc:460] Webserver started at http://127.10.97.126:36949/ using document root <none> and password file <none>
I20260810 00:54:48.407219 1044 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:48.407322 1044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:48.412258 1044 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:54:48.414768 1072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:48.415970 1044 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:54:48.416121 1044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "0cd8f87eb6cc40dd877cda931a8498db"
format_stamp: "Formatted at 2026-08-10 00:54:42 on dist-test-slave-jlzn"
server_key: "9c1d78c30d6bdd8284defd7afb2af23d"
server_key_iv: "1af3faefcd158fa099a3107e7639725f"
server_key_version: "encryptionkey@0"
I20260810 00:54:48.417007 1044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:48.427572 1044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:48.430987 1044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:48.431217 1044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:48.448035 1044 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:43387
I20260810 00:54:48.448071 1143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:43387 every 8 connection(s)
I20260810 00:54:48.449370 1044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:54:48.451216 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 1044
I20260810 00:54:48.452267 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:42033
--local_ip_for_outbound_sockets=127.10.97.65
--tserver_master_addrs=127.10.97.126:43387
--webserver_port=43789
--webserver_interface=127.10.97.65
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:54:48.457677 1144 sys_catalog.cc:263] Verifying existing consensus state
I20260810 00:54:48.461760 1144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db: Bootstrap starting.
I20260810 00:54:48.481287 1144 log.cc:824] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db: Log is configured to *not* fsync() on all Append() calls
I20260810 00:54:48.497952 1144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 00:54:48.498454 1144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db: Bootstrap complete.
I20260810 00:54:48.507118 1144 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd8f87eb6cc40dd877cda931a8498db" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43387 } }
I20260810 00:54:48.508009 1144 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd8f87eb6cc40dd877cda931a8498db, State: Initialized, Role: FOLLOWER
I20260810 00:54:48.508611 1144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd8f87eb6cc40dd877cda931a8498db" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43387 } }
I20260810 00:54:48.508821 1144 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:54:48.508934 1144 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:54:48.509093 1144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 2 FOLLOWER]: Advancing to term 3
I20260810 00:54:48.512717 1144 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd8f87eb6cc40dd877cda931a8498db" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43387 } }
I20260810 00:54:48.513175 1144 leader_election.cc:302] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cd8f87eb6cc40dd877cda931a8498db; no voters:
I20260810 00:54:48.513743 1144 leader_election.cc:288] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 00:54:48.513919 1150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 3 FOLLOWER]: Leader election won for term 3
I20260810 00:54:48.514894 1150 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [term 3 LEADER]: Becoming Leader. State: Replica: 0cd8f87eb6cc40dd877cda931a8498db, State: Running, Role: LEADER
I20260810 00:54:48.515422 1150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd8f87eb6cc40dd877cda931a8498db" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43387 } }
I20260810 00:54:48.515650 1144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:54:48.517836 1153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cd8f87eb6cc40dd877cda931a8498db. Latest consensus state: current_term: 3 leader_uuid: "0cd8f87eb6cc40dd877cda931a8498db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd8f87eb6cc40dd877cda931a8498db" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43387 } } }
I20260810 00:54:48.518204 1153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [sys.catalog]: This master's current role is: LEADER
I20260810 00:54:48.518730 1152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0cd8f87eb6cc40dd877cda931a8498db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd8f87eb6cc40dd877cda931a8498db" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43387 } } }
I20260810 00:54:48.518985 1152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cd8f87eb6cc40dd877cda931a8498db [sys.catalog]: This master's current role is: LEADER
I20260810 00:54:48.526850 1159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:54:48.535316 1159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:54:48.536762 1159 catalog_manager.cc:1259] Loaded cluster ID: fa63f84ed5a1400686fb322b339b7056
I20260810 00:54:48.536854 1159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 00:54:48.538987 1159 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0282a309ad google::LogMessage::Fail() at ??:0
@ 0x7f0282a3388a google::LogMessage::SendToLog() at ??:0
@ 0x7f0282a304de google::LogMessage::Flush() at ??:0
@ 0x7f0282a31a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0298463512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f0298460976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f02984e8610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f02984e8439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f028567c345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f028567f91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f028567f799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f028565aeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f02889346db start_thread at ??:0
@ 0x7f02818ca61f clone at ??:0
W20260810 00:54:48.857251 1146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:48.857502 1146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:48.857594 1146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:48.865094 1146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:48.865217 1146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:48.865399 1146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:54:48.875620 1146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:42033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=43789
--tserver_master_addrs=127.10.97.126:43387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:48.876771 1146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:48.878042 1146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:48.890360 1204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:48.890446 1205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:48.890416 1207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:48.891259 1146 server_base.cc:1034] running on GCE node
I20260810 00:54:48.891924 1146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:48.892789 1146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:48.893968 1146 hybrid_clock.cc:648] HybridClock initialized: now 1786323288893914 us; error 52 us; skew 500 ppm
I20260810 00:54:48.894253 1146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:48 dist-test-slave-jlzn krb5kdc[31862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323288, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:48.897310 1146 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:54:48.898597 1146 webserver.cc:460] Webserver started at http://127.10.97.65:43789/ using document root <none> and password file <none>
I20260810 00:54:48.899212 1146 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:48.899333 1146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:48.904362 1146 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 00:54:48.909044 1217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:48.910269 1146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 00:54:48.910418 1146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "101a6277a18a47aaa8e2a0c52a7cd1da"
format_stamp: "Formatted at 2026-08-10 00:54:43 on dist-test-slave-jlzn"
server_key: "3afb52af1450e397d5818f94648093de"
server_key_iv: "fc629ac818c3b9b356144d02f518ef96"
server_key_version: "encryptionkey@0"
I20260810 00:54:48.911324 1146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:48.924592 1146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:48.928530 1146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:48.928772 1146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:48.929680 1146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:54:48.931375 1232 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260810 00:54:48.936757 1146 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 00:54:48.936877 1146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260810 00:54:48.936988 1146 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 00:54:48.939978 1146 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 00:54:48.940054 1146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260810 00:54:48.940356 1232 tablet_bootstrap.cc:492] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da: Bootstrap starting.
I20260810 00:54:48.965914 1232 log.cc:824] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da: Log is configured to *not* fsync() on all Append() calls
I20260810 00:54:48.976480 1146 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:42033
I20260810 00:54:48.976531 1401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:42033 every 8 connection(s)
I20260810 00:54:48.977667 1146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:54:48.977810 1232 tablet_bootstrap.cc:492] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 00:54:48.978333 1232 tablet_bootstrap.cc:492] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da: Bootstrap complete.
I20260810 00:54:48.979486 1232 ts_tablet_manager.cc:1397] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da: Time spent bootstrapping tablet: real 0.039s user 0.025s sys 0.013s
I20260810 00:54:48.981503 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 1146
I20260810 00:54:48.982260 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:39415
--local_ip_for_outbound_sockets=127.10.97.66
--tserver_master_addrs=127.10.97.126:43387
--webserver_port=35189
--webserver_interface=127.10.97.66
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:54:48.983201 1405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:48.980688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:41225)
0810 00:54:48.981653 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:48.982003 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":406,"threads_started":1}
W20260810 00:54:48.983943 1402 heartbeater.cc:644] Failed to heartbeat to 127.10.97.126:43387 (0 consecutive failures): Network error: Failed to ping master at 127.10.97.126:43387: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
I20260810 00:54:48.984858 1405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:48.984477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:42241)
0810 00:54:48.984636 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:48.984720 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 00:54:48.985870 1405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:48.985480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:48609)
0810 00:54:48.985630 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:48.985721 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 00:54:48.986151 1402 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 00:54:48.987483 1232 raft_consensus.cc:348] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } }
I20260810 00:54:48.988348 1232 raft_consensus.cc:729] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101a6277a18a47aaa8e2a0c52a7cd1da, State: Initialized, Role: FOLLOWER
I20260810 00:54:48.988893 1232 consensus_queue.cc:260] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } }
I20260810 00:54:48.990423 1405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:48.989983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:56795)
0810 00:54:48.990210 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:48.990286 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260810 00:54:48.990444 1232 ts_tablet_manager.cc:1428] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260810 00:54:48.991420 1405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:48.991109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:35625)
0810 00:54:48.991248 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:48.991311 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 00:54:49.304069 1407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:49.304318 1407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:49.304445 1407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:49.311347 1407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:49.311446 1407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:49.311614 1407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:54:49.320827 1407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:39415
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=35189
--tserver_master_addrs=127.10.97.126:43387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:49.321966 1407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:49.323184 1407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:49.335516 1414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:49.335580 1415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:49.335632 1418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:49.336447 1407 server_base.cc:1034] running on GCE node
I20260810 00:54:49.337019 1407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:49.337865 1407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:49.339061 1407 hybrid_clock.cc:648] HybridClock initialized: now 1786323289339009 us; error 40 us; skew 500 ppm
I20260810 00:54:49.339341 1407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:49 dist-test-slave-jlzn krb5kdc[31862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323289, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:49.342239 1407 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:54:49.343585 1407 webserver.cc:460] Webserver started at http://127.10.97.66:35189/ using document root <none> and password file <none>
I20260810 00:54:49.344265 1407 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:49.344383 1407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:49.349436 1407 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260810 00:54:49.351945 1427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:49.353152 1407 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:54:49.353304 1407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "90ed6faf3425442ab2a6249ee33a62c8"
format_stamp: "Formatted at 2026-08-10 00:54:43 on dist-test-slave-jlzn"
server_key: "e080b34aa8b3334c315c0756d892a049"
server_key_iv: "0c529fcc224024c89a889c99edef76f8"
server_key_version: "encryptionkey@0"
I20260810 00:54:49.354228 1407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:49.368855 1407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:49.372349 1407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:49.372586 1407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:49.373526 1407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:54:49.374738 1407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:54:49.374843 1407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:49.374948 1407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:54:49.375022 1407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:49.404407 1407 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:39415
I20260810 00:54:49.404474 1590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:39415 every 8 connection(s)
I20260810 00:54:49.405721 1407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:54:49.407966 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 1407
I20260810 00:54:49.408550 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:34171
--local_ip_for_outbound_sockets=127.10.97.67
--tserver_master_addrs=127.10.97.126:43387
--webserver_port=46697
--webserver_interface=127.10.97.67
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:46479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:54:49.410490 1594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:49.408361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:43533)
0810 00:54:49.409174 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:49.409413 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":317,"threads_started":1}
W20260810 00:54:49.411185 1591 heartbeater.cc:644] Failed to heartbeat to 127.10.97.126:43387 (0 consecutive failures): Network error: Failed to ping master at 127.10.97.126:43387: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
I20260810 00:54:49.412012 1594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:49.411631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:59963)
0810 00:54:49.411782 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:49.411894 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 00:54:49.412869 1594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:49.412528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:35655)
0810 00:54:49.412662 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:49.412744 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 00:54:49.413116 1591 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 00:54:49.728104 1595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:54:49.728358 1595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:54:49.728453 1595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:54:49.735426 1595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:54:49.735517 1595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:54:49.735627 1595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:54:49.744815 1595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:46479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:34171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=46697
--tserver_master_addrs=127.10.97.126:43387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:54:49.745858 1595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:54:49.747030 1595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:54:49.759018 1603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:49.759069 1608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:54:49.759127 1604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:54:49.760113 1595 server_base.cc:1034] running on GCE node
I20260810 00:54:49.760663 1595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:54:49.761472 1595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:54:49.762665 1595 hybrid_clock.cc:648] HybridClock initialized: now 1786323289762622 us; error 36 us; skew 500 ppm
I20260810 00:54:49.762940 1595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:54:49 dist-test-slave-jlzn krb5kdc[31862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323289, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:54:49.765865 1595 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:54:49.767118 1595 webserver.cc:460] Webserver started at http://127.10.97.67:46697/ using document root <none> and password file <none>
I20260810 00:54:49.767730 1595 fs_manager.cc:359] Metadata directory not provided
I20260810 00:54:49.767875 1595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:54:49.772730 1595 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260810 00:54:49.775189 1620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:49.776386 1595 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:54:49.776525 1595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "d833f3e4ccaa4e329cfec01b259b12b5"
format_stamp: "Formatted at 2026-08-10 00:54:44 on dist-test-slave-jlzn"
server_key: "4375b6f8ffe702189e57a4a0feb428a2"
server_key_iv: "63cf1f9f1cbe16c4534ec62f3a7061d1"
server_key_version: "encryptionkey@0"
I20260810 00:54:49.777421 1595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:54:49.803095 1595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:54:49.806452 1595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:54:49.806701 1595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:54:49.807590 1595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:54:49.808827 1595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:54:49.808913 1595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:49.809016 1595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:54:49.809087 1595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:54:49.837070 1595 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:34171
I20260810 00:54:49.837152 1782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:34171 every 8 connection(s)
I20260810 00:54:49.838406 1595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:54:49.842831 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:49.841184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:43337)
0810 00:54:49.841841 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:49.842040 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":247,"threads_started":1}
I20260810 00:54:49.843420 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 1595
W20260810 00:54:49.843483 1783 heartbeater.cc:644] Failed to heartbeat to 127.10.97.126:43387 (0 consecutive failures): Network error: Failed to ping master at 127.10.97.126:43387: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
I20260810 00:54:49.844278 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:49.843955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:51529)
0810 00:54:49.844090 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:49.844166 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 00:54:49.845160 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:49.844802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:52253)
0810 00:54:49.844940 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:49.845026 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 00:54:49.845367 1783 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 00:54:49.992852 1787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:49.992111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:50053)
0810 00:54:49.992562 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:49.992667 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":215,"threads_started":1}
I20260810 00:54:50.265472 1791 raft_consensus.cc:482] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:54:50.265753 1791 raft_consensus.cc:504] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } }
I20260810 00:54:50.267297 1791 leader_election.cc:288] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90ed6faf3425442ab2a6249ee33a62c8 (127.10.97.66:39415), d833f3e4ccaa4e329cfec01b259b12b5 (127.10.97.67:34171)
Aug 10 00:54:50 dist-test-slave-jlzn krb5kdc[31862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323288, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.67@KRBTEST.COM
Aug 10 00:54:50 dist-test-slave-jlzn krb5kdc[31862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323288, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.66@KRBTEST.COM
I20260810 00:54:50.289083 1794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:50.268009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:45069 (local address 127.10.97.67:34171)
0810 00:54:50.269808 (+ 1799us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:50.269820 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:50.269859 (+ 39us) server_negotiation.cc:407] Connection header received
0810 00:54:50.271895 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:50.271905 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:50.272307 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:50.272608 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:50.274749 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:50.276048 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:50.277799 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:50.278410 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:50.278594 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:50.282365 (+ 3771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:50.282375 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:50.282390 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:50.282458 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:50.285454 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:50.286266 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:50.286274 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:50.286281 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:50.286466 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:50.286917 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:50.286922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:50.286925 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:50.287420 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:50.287584 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:50.288364 (+ 780us) server_negotiation.cc:299] Negotiation successful
0810 00:54:50.288711 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1626,"thread_start_us":1453,"threads_started":1}
I20260810 00:54:50.289108 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:50.267629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:34171 (local address 127.10.97.65:45069)
0810 00:54:50.269101 (+ 1472us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:50.269118 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:50.269243 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:50.272577 (+ 3334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:50.272580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:50.273494 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:50.274532 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:50.274559 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:50.276212 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:50.276216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:50.277664 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:50.277678 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:50.278587 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:50.278592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:50.278774 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:50.279470 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:50.279499 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:50.282094 (+ 2595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:50.285587 (+ 3493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:50.285596 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:50.285610 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:50.286142 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:50.286587 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:50.286592 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:50.286595 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:50.286818 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:50.287567 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:50.287574 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:50.287900 (+ 326us) client_negotiation.cc:769] Sending connection context
0810 00:54:50.288136 (+ 236us) client_negotiation.cc:240] Negotiation successful
0810 00:54:50.288651 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1361,"mutex_wait_us":295,"spinlock_wait_cycles":768,"thread_start_us":1008,"threads_started":1}
I20260810 00:54:50.289106 1792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:50.267765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:57673 (local address 127.10.97.66:39415)
0810 00:54:50.269404 (+ 1639us) server_negotiation.cc:206] Beginning negotiation
0810 00:54:50.269420 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 00:54:50.269461 (+ 41us) server_negotiation.cc:407] Connection header received
0810 00:54:50.271531 (+ 2070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:54:50.271543 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:54:50.272062 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:54:50.272419 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:54:50.275936 (+ 3517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:50.277137 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:50.278475 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:54:50.279256 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:54:50.279411 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:50.282685 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:54:50.282692 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:54:50.282703 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:54:50.282735 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:54:50.285757 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:50.286430 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:50.286436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:50.286440 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:50.286647 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:54:50.287038 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:54:50.287043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:54:50.287046 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:54:50.287546 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:54:50.287707 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 00:54:50.288391 (+ 684us) server_negotiation.cc:299] Negotiation successful
0810 00:54:50.288700 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1454,"thread_start_us":1345,"threads_started":1}
I20260810 00:54:50.289171 1787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:50.267594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:39415 (local address 127.10.97.65:57673)
0810 00:54:50.267760 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:50.267834 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0810 00:54:50.268420 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:54:50.272541 (+ 4121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:54:50.272553 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:54:50.273494 (+ 941us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:54:50.274532 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:50.274559 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:50.277291 (+ 2732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:50.277297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:50.278340 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:54:50.278351 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:54:50.279473 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:54:50.279479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:54:50.279637 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:54:50.280148 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:54:50.280171 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:54:50.282445 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:54:50.285878 (+ 3433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:50.285881 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:50.285884 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:50.286311 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:50.286763 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:54:50.286767 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:54:50.286769 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:54:50.286922 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:54:50.287685 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:54:50.287690 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:54:50.287903 (+ 213us) client_negotiation.cc:769] Sending connection context
0810 00:54:50.288136 (+ 233us) client_negotiation.cc:240] Negotiation successful
0810 00:54:50.288784 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":286,"spinlock_wait_cycles":39936}
W20260810 00:54:50.292038 1224 leader_election.cc:341] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d833f3e4ccaa4e329cfec01b259b12b5 (127.10.97.67:34171): Not found: Tablet not found: 099d179eee004843a050f68f9de25f0f
W20260810 00:54:50.292309 1226 leader_election.cc:341] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90ed6faf3425442ab2a6249ee33a62c8 (127.10.97.66:39415): Not found: Tablet not found: 099d179eee004843a050f68f9de25f0f
I20260810 00:54:50.292472 1226 leader_election.cc:302] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 101a6277a18a47aaa8e2a0c52a7cd1da; no voters: 90ed6faf3425442ab2a6249ee33a62c8, d833f3e4ccaa4e329cfec01b259b12b5
I20260810 00:54:50.292815 1791 raft_consensus.cc:2726] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 00:54:50.414373 1795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:50.413534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:51665)
0810 00:54:50.413965 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:50.414119 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":201,"threads_started":1}
I20260810 00:54:50.846509 1796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:50.845812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:60265)
0810 00:54:50.846222 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:50.846327 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":182,"threads_started":1}
I20260810 00:54:50.994277 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:50.993619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:52445)
0810 00:54:50.994026 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:50.994103 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":204,"threads_started":1}
I20260810 00:54:51.415782 1802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:51.415116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:34865)
0810 00:54:51.415537 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:51.415635 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":209,"threads_started":1}
I20260810 00:54:51.847972 1803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:51.847258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:44641)
0810 00:54:51.847682 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:51.847767 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":224,"threads_started":1}
I20260810 00:54:51.995774 1804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:51.995027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:39405)
0810 00:54:51.995506 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:51.995588 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":208,"threads_started":1}
I20260810 00:54:52.287133 1805 raft_consensus.cc:482] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:54:52.287330 1805 raft_consensus.cc:504] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } }
I20260810 00:54:52.287987 1805 leader_election.cc:288] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90ed6faf3425442ab2a6249ee33a62c8 (127.10.97.66:39415), d833f3e4ccaa4e329cfec01b259b12b5 (127.10.97.67:34171)
W20260810 00:54:52.288933 1224 leader_election.cc:341] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d833f3e4ccaa4e329cfec01b259b12b5 (127.10.97.67:34171): Not found: Tablet not found: 099d179eee004843a050f68f9de25f0f
W20260810 00:54:52.289115 1226 leader_election.cc:341] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90ed6faf3425442ab2a6249ee33a62c8 (127.10.97.66:39415): Not found: Tablet not found: 099d179eee004843a050f68f9de25f0f
I20260810 00:54:52.289249 1226 leader_election.cc:302] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 101a6277a18a47aaa8e2a0c52a7cd1da; no voters: 90ed6faf3425442ab2a6249ee33a62c8, d833f3e4ccaa4e329cfec01b259b12b5
I20260810 00:54:52.289498 1805 raft_consensus.cc:2726] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 00:54:52.417171 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:52.416531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:37759)
0810 00:54:52.416936 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:52.417014 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":206,"threads_started":1}
I20260810 00:54:52.849370 1812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:52.848680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:46161)
0810 00:54:52.849137 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:52.849225 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":220,"threads_started":1}
I20260810 00:54:52.997225 1814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:52.996561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:58887)
0810 00:54:52.996974 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:52.997050 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":221,"threads_started":1}
I20260810 00:54:53.418586 1815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:53.417929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:52399)
0810 00:54:53.418336 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:53.418412 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":206,"threads_started":1}
I20260810 00:54:53.850775 1816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:53.850073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:47377)
0810 00:54:53.850491 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:53.850578 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":201,"threads_started":1}
I20260810 00:54:53.998848 1817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:53.998174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:51225)
0810 00:54:53.998586 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:53.998672 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":219,"threads_started":1}
I20260810 00:54:54.359844 1818 raft_consensus.cc:482] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:54:54.360091 1818 raft_consensus.cc:504] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } }
I20260810 00:54:54.360715 1818 leader_election.cc:288] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90ed6faf3425442ab2a6249ee33a62c8 (127.10.97.66:39415), d833f3e4ccaa4e329cfec01b259b12b5 (127.10.97.67:34171)
W20260810 00:54:54.361562 1224 leader_election.cc:341] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d833f3e4ccaa4e329cfec01b259b12b5 (127.10.97.67:34171): Not found: Tablet not found: 099d179eee004843a050f68f9de25f0f
W20260810 00:54:54.361752 1226 leader_election.cc:341] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90ed6faf3425442ab2a6249ee33a62c8 (127.10.97.66:39415): Not found: Tablet not found: 099d179eee004843a050f68f9de25f0f
I20260810 00:54:54.361871 1226 leader_election.cc:302] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 101a6277a18a47aaa8e2a0c52a7cd1da; no voters: 90ed6faf3425442ab2a6249ee33a62c8, d833f3e4ccaa4e329cfec01b259b12b5
I20260810 00:54:54.362115 1818 raft_consensus.cc:2726] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 00:54:54.420022 1819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:54.419353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:50857)
0810 00:54:54.419769 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:54.419872 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":235,"threads_started":1}
I20260810 00:54:54.852257 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:54.851566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:35029)
0810 00:54:54.851991 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:54.852072 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":242,"threads_started":1}
I20260810 00:54:55.000291 1824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:54.999592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:53401)
0810 00:54:55.000031 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:55.000133 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":255,"threads_started":1}
I20260810 00:54:55.421520 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:55.420789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:52199)
0810 00:54:55.421267 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:55.421349 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":258,"threads_started":1}
I20260810 00:54:55.853686 1827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:55.853009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:42261)
0810 00:54:55.853437 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:55.853512 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":228,"threads_started":1}
I20260810 00:54:56.001699 1828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:56.001028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:49163)
0810 00:54:56.001435 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:56.001515 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":212,"threads_started":1}
I20260810 00:54:56.423092 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:56.422317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:51705)
0810 00:54:56.422838 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:56.422914 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":232,"threads_started":1}
I20260810 00:54:56.855250 1832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:56.854441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:43695)
0810 00:54:56.854959 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:56.855038 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":233,"threads_started":1}
I20260810 00:54:57.003134 1834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:57.002482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:34319)
0810 00:54:57.002898 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:57.002984 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":220,"threads_started":1}
I20260810 00:54:57.424525 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:57.423880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:39805)
0810 00:54:57.424271 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:57.424348 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":195,"threads_started":1}
I20260810 00:54:57.497640 1839 raft_consensus.cc:482] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:54:57.497856 1839 raft_consensus.cc:504] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } }
I20260810 00:54:57.498591 1839 leader_election.cc:288] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90ed6faf3425442ab2a6249ee33a62c8 (127.10.97.66:39415), d833f3e4ccaa4e329cfec01b259b12b5 (127.10.97.67:34171)
W20260810 00:54:57.499467 1224 leader_election.cc:341] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d833f3e4ccaa4e329cfec01b259b12b5 (127.10.97.67:34171): Not found: Tablet not found: 099d179eee004843a050f68f9de25f0f
W20260810 00:54:57.499655 1226 leader_election.cc:341] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90ed6faf3425442ab2a6249ee33a62c8 (127.10.97.66:39415): Not found: Tablet not found: 099d179eee004843a050f68f9de25f0f
I20260810 00:54:57.499752 1226 leader_election.cc:302] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 101a6277a18a47aaa8e2a0c52a7cd1da; no voters: 90ed6faf3425442ab2a6249ee33a62c8, d833f3e4ccaa4e329cfec01b259b12b5
I20260810 00:54:57.500002 1839 raft_consensus.cc:2726] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 00:54:57.856748 1841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:57.856102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:38239)
0810 00:54:57.856512 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:57.856587 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":207,"threads_started":1}
I20260810 00:54:58.004580 1842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:58.003928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:45523)
0810 00:54:58.004317 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:58.004396 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":197,"threads_started":1}
I20260810 00:54:58.425894 1844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:58.425249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:50001)
0810 00:54:58.425672 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:58.425749 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":203,"threads_started":1}
I20260810 00:54:58.858198 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:58.857471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:56223)
0810 00:54:58.857939 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:58.858018 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":227,"threads_started":1}
I20260810 00:54:59.006062 1847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:59.005416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:38275)
0810 00:54:59.005836 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:59.005913 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":228,"threads_started":1}
I20260810 00:54:59.427326 1848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:59.426623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:53877)
0810 00:54:59.427061 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:59.427153 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":215,"threads_started":1}
I20260810 00:54:59.859640 1849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:54:59.858980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:49099)
0810 00:54:59.859378 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0810 00:54:59.859456 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":209,"threads_started":1}
I20260810 00:55:00.007457 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:00.006795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:43901)
0810 00:55:00.007198 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:00.007272 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":201,"threads_started":1}
I20260810 00:55:00.428802 1852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:00.428111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:46713)
0810 00:55:00.428559 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:00.428642 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":222,"threads_started":1}
I20260810 00:55:00.861150 1853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:00.860406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:43497)
0810 00:55:00.860894 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:00.860977 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":250,"threads_started":1}
I20260810 00:55:01.008913 1854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:01.008234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:56439)
0810 00:55:01.008653 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:01.008731 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":226,"threads_started":1}
I20260810 00:55:01.430241 1858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:01.429564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:51903)
0810 00:55:01.429983 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:01.430060 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":221,"threads_started":1}
I20260810 00:55:01.862551 1860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:01.861881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:40951)
0810 00:55:01.862310 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:01.862386 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":207,"threads_started":1}
I20260810 00:55:02.010418 1863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:02.009658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:59589)
0810 00:55:02.010075 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:02.010191 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":201,"threads_started":1}
I20260810 00:55:02.431720 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:02.431016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:35519)
0810 00:55:02.431441 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:02.431546 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":217,"threads_started":1}
I20260810 00:55:02.864423 1866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:02.863366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:38633)
0810 00:55:02.864074 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:02.864219 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":322,"threads_started":1}
I20260810 00:55:03.011924 1867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:03.011212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:60485)
0810 00:55:03.011636 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:03.011721 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":208,"threads_started":1}
I20260810 00:55:03.433176 1868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:03.432476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:52753)
0810 00:55:03.432899 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:03.432988 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":226,"threads_started":1}
I20260810 00:55:03.865949 1870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:03.865265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:44141)
0810 00:55:03.865682 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:03.865773 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":213,"threads_started":1}
I20260810 00:55:04.013582 1871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:04.012889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.65:38193)
0810 00:55:04.013330 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:04.013420 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":204,"threads_started":1}
I20260810 00:55:04.434926 1873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:04.434195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.66:59283)
0810 00:55:04.434686 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:04.434762 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":259,"threads_started":1}
I20260810 00:55:04.465595 1875 raft_consensus.cc:482] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:55:04.465807 1875 raft_consensus.cc:504] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101a6277a18a47aaa8e2a0c52a7cd1da" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 42033 } } peers { permanent_uuid: "90ed6faf3425442ab2a6249ee33a62c8" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 39415 } } peers { permanent_uuid: "d833f3e4ccaa4e329cfec01b259b12b5" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34171 } }
I20260810 00:55:04.466665 1875 leader_election.cc:288] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90ed6faf3425442ab2a6249ee33a62c8 (127.10.97.66:39415), d833f3e4ccaa4e329cfec01b259b12b5 (127.10.97.67:34171)
W20260810 00:55:04.467410 1226 leader_election.cc:341] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90ed6faf3425442ab2a6249ee33a62c8 (127.10.97.66:39415): Not found: Tablet not found: 099d179eee004843a050f68f9de25f0f
W20260810 00:55:04.467582 1224 leader_election.cc:341] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d833f3e4ccaa4e329cfec01b259b12b5 (127.10.97.67:34171): Not found: Tablet not found: 099d179eee004843a050f68f9de25f0f
I20260810 00:55:04.467681 1224 leader_election.cc:302] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 101a6277a18a47aaa8e2a0c52a7cd1da; no voters: 90ed6faf3425442ab2a6249ee33a62c8, d833f3e4ccaa4e329cfec01b259b12b5
I20260810 00:55:04.467919 1875 raft_consensus.cc:2726] T 099d179eee004843a050f68f9de25f0f P 101a6277a18a47aaa8e2a0c52a7cd1da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 00:55:04.853950 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 1146
I20260810 00:55:04.867825 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:04.866967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43387 (local address 127.10.97.67:36727)
0810 00:55:04.867464 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:04.867607 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":240,"threads_started":1}
I20260810 00:55:04.870781 1876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:04.854947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49956 (local address 127.10.97.65:42033)
0810 00:55:04.855437 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:04.855447 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:04.855483 (+ 36us) server_negotiation.cc:407] Connection header received
0810 00:55:04.855569 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:04.855578 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:04.855920 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:04.856081 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:04.858497 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:04.859544 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:04.860861 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:04.861417 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:04.861516 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:04.864233 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:04.864249 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:04.864267 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:04.864305 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:04.867677 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:04.868639 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:04.868648 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:04.868652 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:04.868839 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:04.869281 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:04.869287 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:04.869290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:04.869816 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:04.870101 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:04.870310 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 00:55:04.870511 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":701,"threads_started":1}
I20260810 00:55:04.872040 1396 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:55:04.959681 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 1146
I20260810 00:55:04.981303 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 1407
I20260810 00:55:04.994091 1879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:04.982030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57088 (local address 127.10.97.66:39415)
0810 00:55:04.982521 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:04.982525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:04.982542 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:55:04.982602 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:04.982606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:04.982674 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:04.982763 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:04.984199 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:04.984987 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:04.986142 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:04.986549 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:04.986639 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:04.989201 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:04.989205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:04.989209 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:04.989241 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:04.991878 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:04.992452 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:04.992455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:04.992457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:04.992604 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:04.992872 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:04.992875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:04.992877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:04.993119 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:04.993324 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:04.993541 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 00:55:04.993734 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":276,"threads_started":1}
I20260810 00:55:04.994771 1585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:55:05.077404 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 1407
I20260810 00:55:05.101372 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 1595
I20260810 00:55:05.114519 1881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:05.102173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50684 (local address 127.10.97.67:34171)
0810 00:55:05.102732 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:05.102737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:05.102754 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:55:05.102811 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:05.102815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:05.102883 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:05.102977 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:05.104419 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:05.105203 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:05.106361 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:05.106805 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:05.106915 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:05.109538 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:05.109542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:05.109546 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:05.109578 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:05.111940 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:05.112610 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:05.112616 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:05.112620 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:05.112762 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:05.113118 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:05.113122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:05.113126 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:05.113378 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:05.113653 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:05.113933 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 00:55:05.114153 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":281,"threads_started":1}
I20260810 00:55:05.115206 1775 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:55:05.198218 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 1595
2026-08-10T00:55:05Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24252 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:55:05 dist-test-slave-jlzn krb5kdc[1886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:55:05 dist-test-slave-jlzn krb5kdc[1886](info): set up 2 sockets
Aug 10 00:55:05 dist-test-slave-jlzn krb5kdc[1886](info): commencing operation
krb5kdc: starting...
W20260810 00:55:06.401640 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:55:06 dist-test-slave-jlzn krb5kdc[1886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:55:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:55:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:55:06.640477 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:42075
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:39661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:42075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:55:06.966276 1906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:55:06.966553 1906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:55:06.966683 1906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:55:06.973690 1906 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 00:55:06.973778 1906 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 00:55:06.973847 1906 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 00:55:06.973920 1906 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:55:06.973995 1906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:55:06.974088 1906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:55:06.974149 1906 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:55:06.974225 1906 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:55:06.983565 1906 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:42075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:42075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:55:06.984737 1906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:55:06.986006 1906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:55:06.996937 1914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:06.997005 1916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:06.997121 1918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:55:06.998029 1906 server_base.cc:1034] running on GCE node
I20260810 00:55:06.998793 1906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:55:07.000121 1906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:55:07.001339 1906 hybrid_clock.cc:648] HybridClock initialized: now 1786323307001284 us; error 32 us; skew 500 ppm
I20260810 00:55:07.001629 1906 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:55:07 dist-test-slave-jlzn krb5kdc[1886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323307, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:55:07.004588 1906 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:55:07.005836 1906 webserver.cc:460] Webserver started at http://127.10.97.126:34067/ using document root <none> and password file <none>
I20260810 00:55:07.006434 1906 fs_manager.cc:359] Metadata directory not provided
I20260810 00:55:07.006520 1906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:55:07.006775 1906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:55:07.009600 1906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "8cc21197f71449078605f8078a16825e"
format_stamp: "Formatted at 2026-08-10 00:55:07 on dist-test-slave-jlzn"
server_key: "2221b4e1457a6a13623b621ffe498eb9"
server_key_iv: "197294f61c048fd25c902d27aa5528ea"
server_key_version: "encryptionkey@0"
I20260810 00:55:07.010306 1906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "8cc21197f71449078605f8078a16825e"
format_stamp: "Formatted at 2026-08-10 00:55:07 on dist-test-slave-jlzn"
server_key: "2221b4e1457a6a13623b621ffe498eb9"
server_key_iv: "197294f61c048fd25c902d27aa5528ea"
server_key_version: "encryptionkey@0"
I20260810 00:55:07.015007 1906 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 00:55:07.018170 1932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:07.019366 1906 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:55:07.019516 1906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "8cc21197f71449078605f8078a16825e"
format_stamp: "Formatted at 2026-08-10 00:55:07 on dist-test-slave-jlzn"
server_key: "2221b4e1457a6a13623b621ffe498eb9"
server_key_iv: "197294f61c048fd25c902d27aa5528ea"
server_key_version: "encryptionkey@0"
I20260810 00:55:07.019688 1906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:55:07.037739 1906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:55:07.041245 1906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:55:07.041448 1906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:55:07.058265 1906 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:42075
I20260810 00:55:07.058272 2010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:42075 every 8 connection(s)
I20260810 00:55:07.059442 1906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:55:07.060316 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 1906
I20260810 00:55:07.060518 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:55:07.061389 10629 external_mini_cluster.cc:1428] Setting key 080b9ecb6f50403948114835d463a493
I20260810 00:55:07.064188 2014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 00:55:07 dist-test-slave-jlzn krb5kdc[1886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:55:07.074255 2014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e: Bootstrap starting.
I20260810 00:55:07.077145 2014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e: Neither blocks nor log segments found. Creating new log.
I20260810 00:55:07.078147 2014 log.cc:824] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e: Log is configured to *not* fsync() on all Append() calls
I20260810 00:55:07.080706 2014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e: No bootstrap required, opened a new log
I20260810 00:55:07.081936 2017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:07.063336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58550 (local address 127.10.97.126:42075)
0810 00:55:07.064058 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:07.064069 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:07.064108 (+ 39us) server_negotiation.cc:407] Connection header received
0810 00:55:07.065699 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:07.065720 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:07.066050 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:07.066389 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:07.067864 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:07.069158 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:07.070538 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:07.071205 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:07.071379 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:07.075235 (+ 3856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:07.075243 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:07.075255 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:07.075292 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:07.078184 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:07.078802 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:07.078808 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:07.078813 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:07.078988 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:07.079378 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:07.079381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:07.079383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:07.079887 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:07.080166 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:07.080602 (+ 436us) server_negotiation.cc:299] Negotiation successful
0810 00:55:07.081016 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":309,"threads_started":1}
I20260810 00:55:07.087749 2014 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc21197f71449078605f8078a16825e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42075 } }
I20260810 00:55:07.088110 2014 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:55:07.088205 2014 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cc21197f71449078605f8078a16825e, State: Initialized, Role: FOLLOWER
I20260810 00:55:07.088714 2014 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc21197f71449078605f8078a16825e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42075 } }
I20260810 00:55:07.088922 2014 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:55:07.089046 2014 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:55:07.089200 2014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:55:07.091130 2014 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc21197f71449078605f8078a16825e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42075 } }
I20260810 00:55:07.091569 2014 leader_election.cc:302] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cc21197f71449078605f8078a16825e; no voters:
I20260810 00:55:07.092131 2014 leader_election.cc:288] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:55:07.092298 2019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:55:07.092940 2019 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 1 LEADER]: Becoming Leader. State: Replica: 8cc21197f71449078605f8078a16825e, State: Running, Role: LEADER
I20260810 00:55:07.093436 2019 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc21197f71449078605f8078a16825e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42075 } }
I20260810 00:55:07.093945 2014 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:55:07.096130 2021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cc21197f71449078605f8078a16825e. Latest consensus state: current_term: 1 leader_uuid: "8cc21197f71449078605f8078a16825e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc21197f71449078605f8078a16825e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42075 } } }
I20260810 00:55:07.096516 2021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [sys.catalog]: This master's current role is: LEADER
I20260810 00:55:07.097219 2020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cc21197f71449078605f8078a16825e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc21197f71449078605f8078a16825e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42075 } } }
I20260810 00:55:07.097496 2020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [sys.catalog]: This master's current role is: LEADER
I20260810 00:55:07.101022 2028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:55:07.105417 2028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:55:07.112687 2028 catalog_manager.cc:1347] Generated new cluster ID: b3a7e13690c849189952149cd0196486
I20260810 00:55:07.112792 2028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:55:07.127103 2028 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:55:07.128283 2028 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:55:07.134445 2028 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e: Generated new TSK 0
I20260810 00:55:07.135120 2028 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
I20260810 00:55:07.200837 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:42075
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:39661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:55:07.515627 2043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:55:07.515897 2043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:55:07.515993 2043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:55:07.522804 2043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:55:07.522899 2043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:55:07.523008 2043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:55:07.532102 2043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:42075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:55:07.533186 2043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:55:07.534375 2043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:55:07.546312 2050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:07.546365 2051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:07.546381 2054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:55:07.547179 2043 server_base.cc:1034] running on GCE node
I20260810 00:55:07.547691 2043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:55:07.548539 2043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:55:07.549721 2043 hybrid_clock.cc:648] HybridClock initialized: now 1786323307549665 us; error 47 us; skew 500 ppm
I20260810 00:55:07.550016 2043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:55:07 dist-test-slave-jlzn krb5kdc[1886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323307, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:55:07.552933 2043 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:55:07.554955 2043 webserver.cc:460] Webserver started at http://127.10.97.65:46255/ using document root <none> and password file <none>
I20260810 00:55:07.555794 2043 fs_manager.cc:359] Metadata directory not provided
I20260810 00:55:07.555956 2043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:55:07.556315 2043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:55:07.559167 2043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "a65d29f653b84fa7a7dbb52e69b3f19f"
format_stamp: "Formatted at 2026-08-10 00:55:07 on dist-test-slave-jlzn"
server_key: "b1b4613a97c41d82fe0b450fc5704b0a"
server_key_iv: "746e341fbb51fc81a6285609de39174e"
server_key_version: "encryptionkey@0"
I20260810 00:55:07.559942 2043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "a65d29f653b84fa7a7dbb52e69b3f19f"
format_stamp: "Formatted at 2026-08-10 00:55:07 on dist-test-slave-jlzn"
server_key: "b1b4613a97c41d82fe0b450fc5704b0a"
server_key_iv: "746e341fbb51fc81a6285609de39174e"
server_key_version: "encryptionkey@0"
I20260810 00:55:07.564733 2043 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 00:55:07.567986 2064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:07.569150 2043 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:55:07.569303 2043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "a65d29f653b84fa7a7dbb52e69b3f19f"
format_stamp: "Formatted at 2026-08-10 00:55:07 on dist-test-slave-jlzn"
server_key: "b1b4613a97c41d82fe0b450fc5704b0a"
server_key_iv: "746e341fbb51fc81a6285609de39174e"
server_key_version: "encryptionkey@0"
I20260810 00:55:07.569468 2043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:55:07.581284 2043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:55:07.584821 2043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:55:07.585075 2043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:55:07.585904 2043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:55:07.587082 2043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:55:07.587169 2043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:07.587285 2043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:55:07.587344 2043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:07.617054 2043 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:34371
I20260810 00:55:07.617101 2248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:34371 every 8 connection(s)
I20260810 00:55:07.618191 2043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:55:07.621273 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 2043
I20260810 00:55:07.621501 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:55:07.622383 10629 external_mini_cluster.cc:1428] Setting key 9b9e4b10bdee37a8d4216f25ef5a6120
Aug 10 00:55:07 dist-test-slave-jlzn krb5kdc[1886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323307, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:55:07.638095 2252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:07.620462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:52477 (local address 127.10.97.126:42075)
0810 00:55:07.620889 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:07.620895 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:07.621846 (+ 951us) server_negotiation.cc:407] Connection header received
0810 00:55:07.624130 (+ 2284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:07.624135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:07.624198 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:07.624304 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:07.625759 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:07.626771 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:07.628300 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:07.628819 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:07.628941 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:07.632253 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:07.632259 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:07.632263 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:07.632299 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:07.635269 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:07.635945 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:07.635950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:07.635952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:07.636092 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:07.636515 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:07.636519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:07.636522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:07.636839 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:07.636952 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:07.637680 (+ 728us) server_negotiation.cc:299] Negotiation successful
0810 00:55:07.637838 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":220,"threads_started":1}
I20260810 00:55:07.639288 2255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:07.620874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:52477)
0810 00:55:07.621666 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:07.621730 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 00:55:07.623862 (+ 2132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:55:07.624546 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:55:07.624557 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:55:07.625047 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:55:07.625588 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:07.625605 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:07.626893 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:07.626897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:07.628168 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:07.628180 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:07.628994 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:07.628998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:07.629194 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:07.629787 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:55:07.629807 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:55:07.632040 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:55:07.635398 (+ 3358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:55:07.635404 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:55:07.635415 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:55:07.635831 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:55:07.636193 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:55:07.636197 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:55:07.636201 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:55:07.636410 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:55:07.636955 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:55:07.636962 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:55:07.637345 (+ 383us) client_negotiation.cc:769] Sending connection context
0810 00:55:07.637614 (+ 269us) client_negotiation.cc:240] Negotiation successful
0810 00:55:07.638095 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":293,"threads_started":1}
I20260810 00:55:07.640794 2249 heartbeater.cc:347] Connected to a master server at 127.10.97.126:42075
I20260810 00:55:07.641074 2249 heartbeater.cc:464] Registering TS with master...
I20260810 00:55:07.641752 2249 heartbeater.cc:511] Master 127.10.97.126:42075 requested a full tablet report, sending...
I20260810 00:55:07.643692 1958 ts_manager.cc:194] Registered new tserver with Master: a65d29f653b84fa7a7dbb52e69b3f19f (127.10.97.65:34371)
I20260810 00:55:07.645123 1958 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:52477
WARNING: no policy specified for kudu/127.10.97.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
I20260810 00:55:07.681115 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:42075
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:39661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:55:07.997442 2258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:55:07.997718 2258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:55:07.997860 2258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:55:08.004884 2258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:55:08.004993 2258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:55:08.005146 2258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:55:08.014362 2258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:42075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:55:08.015570 2258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:55:08.016849 2258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:55:08.029178 2265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:08.029227 2268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:08.029264 2271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:55:08.029968 2258 server_base.cc:1034] running on GCE node
I20260810 00:55:08.030511 2258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:55:08.031337 2258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:55:08.032524 2258 hybrid_clock.cc:648] HybridClock initialized: now 1786323308032474 us; error 37 us; skew 500 ppm
I20260810 00:55:08.032810 2258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:55:08 dist-test-slave-jlzn krb5kdc[1886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323308, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:55:08.039638 2258 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:55:08.040985 2258 webserver.cc:460] Webserver started at http://127.10.97.66:44847/ using document root <none> and password file <none>
I20260810 00:55:08.041599 2258 fs_manager.cc:359] Metadata directory not provided
I20260810 00:55:08.041716 2258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:55:08.041993 2258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:55:08.044770 2258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "3157f17735584b71a3eea3557e73df11"
format_stamp: "Formatted at 2026-08-10 00:55:08 on dist-test-slave-jlzn"
server_key: "338825120f9c5a5043fba4bd0167b527"
server_key_iv: "325b92fd55f7e903ed4f64d6a4fd34b4"
server_key_version: "encryptionkey@0"
I20260810 00:55:08.045501 2258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "3157f17735584b71a3eea3557e73df11"
format_stamp: "Formatted at 2026-08-10 00:55:08 on dist-test-slave-jlzn"
server_key: "338825120f9c5a5043fba4bd0167b527"
server_key_iv: "325b92fd55f7e903ed4f64d6a4fd34b4"
server_key_version: "encryptionkey@0"
I20260810 00:55:08.050208 2258 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:55:08.053375 2281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:08.054498 2258 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:55:08.054661 2258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "3157f17735584b71a3eea3557e73df11"
format_stamp: "Formatted at 2026-08-10 00:55:08 on dist-test-slave-jlzn"
server_key: "338825120f9c5a5043fba4bd0167b527"
server_key_iv: "325b92fd55f7e903ed4f64d6a4fd34b4"
server_key_version: "encryptionkey@0"
I20260810 00:55:08.054833 2258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:55:08.073738 2258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:55:08.077279 2258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:55:08.077526 2258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:55:08.078358 2258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:55:08.079536 2258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:55:08.079624 2258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:08.079747 2258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:55:08.079818 2258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:08.109651 2258 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:44591
I20260810 00:55:08.109704 2446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:44591 every 8 connection(s)
I20260810 00:55:08.110829 2258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:55:08.111527 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 2258
I20260810 00:55:08.111727 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:55:08.112408 10629 external_mini_cluster.cc:1428] Setting key 19a20f3825b6707a69d18e972b4d9f0d
Aug 10 00:55:08 dist-test-slave-jlzn krb5kdc[1886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323308, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:55:08.131539 2252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:08.113469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:32895 (local address 127.10.97.126:42075)
0810 00:55:08.113656 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:08.113660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:08.114961 (+ 1301us) server_negotiation.cc:407] Connection header received
0810 00:55:08.117346 (+ 2385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:08.117351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:08.117415 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:08.117496 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:08.119259 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.119988 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.121992 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.122418 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.122511 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:08.126167 (+ 3656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:08.126172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:08.126176 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:08.126210 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:08.128704 (+ 2494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:08.129404 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:08.129410 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:08.129412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:08.129541 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:08.129942 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:08.129947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:08.129949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:08.130213 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:08.130346 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:08.130977 (+ 631us) server_negotiation.cc:299] Negotiation successful
0810 00:55:08.131199 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 00:55:08.132493 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:08.113957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:32895)
0810 00:55:08.114780 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:08.114833 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 00:55:08.117067 (+ 2234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:55:08.117745 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:55:08.117755 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:55:08.118395 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:55:08.119110 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:08.119132 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.120122 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.120127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:08.121866 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:08.121881 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.122651 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.122658 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:08.122947 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:08.123588 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:55:08.123622 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:55:08.125965 (+ 2343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:55:08.128833 (+ 2868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:55:08.128842 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:55:08.128856 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:55:08.129289 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:55:08.129646 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:55:08.129651 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:55:08.129655 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:55:08.129836 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:55:08.130337 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:55:08.130344 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:55:08.130638 (+ 294us) client_negotiation.cc:769] Sending connection context
0810 00:55:08.130869 (+ 231us) client_negotiation.cc:240] Negotiation successful
0810 00:55:08.131357 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":324,"threads_started":1}
I20260810 00:55:08.133800 2447 heartbeater.cc:347] Connected to a master server at 127.10.97.126:42075
I20260810 00:55:08.134152 2447 heartbeater.cc:464] Registering TS with master...
I20260810 00:55:08.134995 2447 heartbeater.cc:511] Master 127.10.97.126:42075 requested a full tablet report, sending...
I20260810 00:55:08.136615 1958 ts_manager.cc:194] Registered new tserver with Master: 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591)
I20260810 00:55:08.137670 1958 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:32895
WARNING: no policy specified for kudu/127.10.97.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
I20260810 00:55:08.174669 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:42075
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:39661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:55:08.491894 2452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:55:08.492213 2452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:55:08.492360 2452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:55:08.499382 2452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:55:08.499492 2452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:55:08.499643 2452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:55:08.508795 2452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:42075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:55:08.509963 2452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:55:08.511231 2452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:55:08.523331 2459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:08.523399 2462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:08.523427 2465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:55:08.524809 2452 server_base.cc:1034] running on GCE node
I20260810 00:55:08.525369 2452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:55:08.526258 2452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:55:08.527451 2452 hybrid_clock.cc:648] HybridClock initialized: now 1786323308527402 us; error 41 us; skew 500 ppm
I20260810 00:55:08.527719 2452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:55:08 dist-test-slave-jlzn krb5kdc[1886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323308, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:55:08.530650 2452 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:55:08.531973 2452 webserver.cc:460] Webserver started at http://127.10.97.67:46395/ using document root <none> and password file <none>
I20260810 00:55:08.532589 2452 fs_manager.cc:359] Metadata directory not provided
I20260810 00:55:08.532693 2452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:55:08.532931 2452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:55:08.535640 2452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "5fefe09513824e63b56e6a319338a3ab"
format_stamp: "Formatted at 2026-08-10 00:55:08 on dist-test-slave-jlzn"
server_key: "80d18bd3b04e11902af9621ab7186443"
server_key_iv: "47a67ddd023a7ac190e2783ae6bfe122"
server_key_version: "encryptionkey@0"
I20260810 00:55:08.536507 2452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "5fefe09513824e63b56e6a319338a3ab"
format_stamp: "Formatted at 2026-08-10 00:55:08 on dist-test-slave-jlzn"
server_key: "80d18bd3b04e11902af9621ab7186443"
server_key_iv: "47a67ddd023a7ac190e2783ae6bfe122"
server_key_version: "encryptionkey@0"
I20260810 00:55:08.541368 2452 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 00:55:08.544445 2479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:08.545627 2452 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:55:08.545775 2452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "5fefe09513824e63b56e6a319338a3ab"
format_stamp: "Formatted at 2026-08-10 00:55:08 on dist-test-slave-jlzn"
server_key: "80d18bd3b04e11902af9621ab7186443"
server_key_iv: "47a67ddd023a7ac190e2783ae6bfe122"
server_key_version: "encryptionkey@0"
I20260810 00:55:08.545943 2452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:55:08.565500 2452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:55:08.569316 2452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:55:08.569582 2452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:55:08.570446 2452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:55:08.571702 2452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:55:08.571781 2452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:08.571935 2452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:55:08.571992 2452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:08.602005 2452 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:37729
I20260810 00:55:08.602073 2641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:37729 every 8 connection(s)
I20260810 00:55:08.603137 2452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:55:08.605293 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 2452
I20260810 00:55:08.605487 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:55:08.605970 10629 external_mini_cluster.cc:1428] Setting key aafba1f99a643bba00d348309d324e69
Aug 10 00:55:08 dist-test-slave-jlzn krb5kdc[1886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323308, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:55:08.622020 2252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:08.605431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:40791 (local address 127.10.97.126:42075)
0810 00:55:08.605626 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:08.605632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:08.606839 (+ 1207us) server_negotiation.cc:407] Connection header received
0810 00:55:08.608809 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:08.608815 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:08.608934 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:08.609042 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:08.610336 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.611059 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.612568 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.612977 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.613077 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:08.616160 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:08.616165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:08.616168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:08.616201 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:08.618682 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:08.619495 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:08.619500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:08.619503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:08.619658 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:08.620201 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:08.620206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:08.620209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:08.620608 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:08.620757 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:08.621500 (+ 743us) server_negotiation.cc:299] Negotiation successful
0810 00:55:08.621743 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 00:55:08.622709 2644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:08.605908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:40791)
0810 00:55:08.606674 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:08.606719 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 00:55:08.608538 (+ 1819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:55:08.609219 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:55:08.609227 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:55:08.609688 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:55:08.610209 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:08.610226 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.611172 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.611175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:08.612427 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:08.612439 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.613177 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.613182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:08.613362 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:08.613931 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:55:08.613957 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:55:08.615976 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:55:08.618819 (+ 2843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:55:08.618827 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:55:08.618839 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:55:08.619378 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:55:08.619769 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:55:08.619774 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:55:08.619778 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:55:08.620050 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:55:08.620763 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:55:08.620771 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:55:08.621157 (+ 386us) client_negotiation.cc:769] Sending connection context
0810 00:55:08.621369 (+ 212us) client_negotiation.cc:240] Negotiation successful
0810 00:55:08.621712 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":313,"threads_started":1}
I20260810 00:55:08.623795 2642 heartbeater.cc:347] Connected to a master server at 127.10.97.126:42075
I20260810 00:55:08.624109 2642 heartbeater.cc:464] Registering TS with master...
I20260810 00:55:08.624882 2642 heartbeater.cc:511] Master 127.10.97.126:42075 requested a full tablet report, sending...
I20260810 00:55:08.626166 1958 ts_manager.cc:194] Registered new tserver with Master: 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729)
I20260810 00:55:08.626956 1958 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:40791
I20260810 00:55:08.632998 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:55:08.647764 2249 heartbeater.cc:503] Master 127.10.97.126:42075 was elected leader, sending a full tablet report...
I20260810 00:55:08.649045 2252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:08.637433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58566 (local address 127.10.97.126:42075)
0810 00:55:08.637577 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:08.637581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:08.638098 (+ 517us) server_negotiation.cc:407] Connection header received
0810 00:55:08.638228 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:08.638231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:08.638300 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:08.638377 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:08.639710 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.640422 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.641856 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.642272 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.642361 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:08.644448 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:08.644452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:08.644455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:08.644484 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:08.646612 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:08.647190 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:08.647194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:08.647196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:08.647310 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:08.647620 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:08.647632 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:08.647635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:08.648049 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:08.648259 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:08.648592 (+ 333us) server_negotiation.cc:299] Negotiation successful
0810 00:55:08.648752 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 00:55:08.652781 1958 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58566:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:55:08.659400 1958 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:55:08.682111 2660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:08.670413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53420 (local address 127.10.97.65:34371)
0810 00:55:08.674045 (+ 3632us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:08.674053 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:08.674082 (+ 29us) server_negotiation.cc:407] Connection header received
0810 00:55:08.674177 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:08.674185 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:08.674418 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:08.674610 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:55:08.675890 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.677134 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.679532 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.680975 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.681104 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:08.681325 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:08.681621 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 00:55:08.681793 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3466,"thread_start_us":3268,"threads_started":1}
I20260810 00:55:08.682363 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:08.670357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:37729 (local address 127.0.0.1:44396)
0810 00:55:08.673610 (+ 3253us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:08.673627 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:55:08.673741 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:55:08.675149 (+ 1408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:55:08.675154 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:55:08.675178 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:55:08.675936 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:08.675954 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.678021 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.678025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:08.679995 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:08.680008 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.681508 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.681515 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:08.681681 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:08.681713 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 00:55:08.681781 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 00:55:08.681888 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3147,"mutex_wait_us":2,"thread_start_us":395,"threads_started":1}
I20260810 00:55:08.682358 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:08.670074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34371 (local address 127.0.0.1:53420)
0810 00:55:08.671745 (+ 1671us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:08.671784 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 00:55:08.671961 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:55:08.674655 (+ 2694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:55:08.674662 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:55:08.674691 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:55:08.675625 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:08.675642 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.677256 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.677261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:08.679348 (+ 2087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:08.679364 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.681186 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.681193 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:08.681376 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:08.681426 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 00:55:08.681569 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 00:55:08.681738 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1440,"mutex_wait_us":63,"thread_start_us":2588,"threads_started":1}
I20260810 00:55:08.683094 2662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:08.670996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44396 (local address 127.10.97.67:37729)
0810 00:55:08.674343 (+ 3347us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:08.674348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:08.674374 (+ 26us) server_negotiation.cc:407] Connection header received
0810 00:55:08.674444 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:08.674449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:08.674641 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:08.675156 (+ 515us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:55:08.676160 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.677841 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.680150 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.681024 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.681144 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:08.681367 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:08.682288 (+ 921us) server_negotiation.cc:299] Negotiation successful
0810 00:55:08.682453 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3216,"thread_start_us":3052,"threads_started":1}
I20260810 00:55:08.683457 2659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:08.670898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:44591 (local address 127.0.0.1:46588)
0810 00:55:08.674038 (+ 3140us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:08.674066 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 00:55:08.674176 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:55:08.674883 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:55:08.674888 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:55:08.674917 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:55:08.676108 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:08.676121 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.677980 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.677986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:08.680291 (+ 2305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:08.680306 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.682686 (+ 2380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.682696 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:08.682851 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:08.682883 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 00:55:08.682959 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 00:55:08.683073 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3025,"mutex_wait_us":132,"spinlock_wait_cycles":6528,"thread_start_us":2481,"threads_started":1}
I20260810 00:55:08.684120 2663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:08.671414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46588 (local address 127.10.97.66:44591)
0810 00:55:08.673205 (+ 1791us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:08.673215 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:08.674282 (+ 1067us) server_negotiation.cc:407] Connection header received
0810 00:55:08.674413 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:08.674419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:08.674623 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:08.674802 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:55:08.676268 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.677827 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.680481 (+ 2654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.681797 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.681928 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:08.682098 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:08.683696 (+ 1598us) server_negotiation.cc:299] Negotiation successful
0810 00:55:08.683856 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1619,"thread_start_us":1514,"threads_started":1}
I20260810 00:55:08.686195 2149 tablet_service.cc:1467] Processing CreateTablet for tablet 98ae78a83ded4ff9933051c5d9a25c16 (DEFAULT_TABLE table=test-table [id=33bbe251fa8e457fb550e68a7fd79efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:55:08.686445 2546 tablet_service.cc:1467] Processing CreateTablet for tablet 98ae78a83ded4ff9933051c5d9a25c16 (DEFAULT_TABLE table=test-table [id=33bbe251fa8e457fb550e68a7fd79efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:55:08.687459 2149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98ae78a83ded4ff9933051c5d9a25c16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:55:08.687914 2546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98ae78a83ded4ff9933051c5d9a25c16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:55:08.687465 2349 tablet_service.cc:1467] Processing CreateTablet for tablet 98ae78a83ded4ff9933051c5d9a25c16 (DEFAULT_TABLE table=test-table [id=33bbe251fa8e457fb550e68a7fd79efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:55:08.688704 2349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98ae78a83ded4ff9933051c5d9a25c16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:55:08.698791 2664 tablet_bootstrap.cc:492] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f: Bootstrap starting.
I20260810 00:55:08.702068 2665 tablet_bootstrap.cc:492] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab: Bootstrap starting.
I20260810 00:55:08.703080 2664 tablet_bootstrap.cc:654] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f: Neither blocks nor log segments found. Creating new log.
I20260810 00:55:08.704054 2666 tablet_bootstrap.cc:492] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11: Bootstrap starting.
I20260810 00:55:08.704530 2664 log.cc:824] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f: Log is configured to *not* fsync() on all Append() calls
I20260810 00:55:08.705438 2665 tablet_bootstrap.cc:654] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab: Neither blocks nor log segments found. Creating new log.
I20260810 00:55:08.706480 2665 log.cc:824] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab: Log is configured to *not* fsync() on all Append() calls
I20260810 00:55:08.706936 2666 tablet_bootstrap.cc:654] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11: Neither blocks nor log segments found. Creating new log.
I20260810 00:55:08.707599 2664 tablet_bootstrap.cc:492] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f: No bootstrap required, opened a new log
I20260810 00:55:08.707873 2664 ts_tablet_manager.cc:1397] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260810 00:55:08.708128 2666 log.cc:824] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11: Log is configured to *not* fsync() on all Append() calls
I20260810 00:55:08.709465 2665 tablet_bootstrap.cc:492] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab: No bootstrap required, opened a new log
I20260810 00:55:08.709717 2665 ts_tablet_manager.cc:1397] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260810 00:55:08.710346 2666 tablet_bootstrap.cc:492] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11: No bootstrap required, opened a new log
I20260810 00:55:08.710568 2666 ts_tablet_manager.cc:1397] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 00:55:08.717326 2666 raft_consensus.cc:348] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:08.717794 2666 raft_consensus.cc:374] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:55:08.717940 2666 raft_consensus.cc:729] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3157f17735584b71a3eea3557e73df11, State: Initialized, Role: FOLLOWER
I20260810 00:55:08.717870 2664 raft_consensus.cc:348] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:08.718287 2664 raft_consensus.cc:374] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:55:08.718413 2664 raft_consensus.cc:729] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a65d29f653b84fa7a7dbb52e69b3f19f, State: Initialized, Role: FOLLOWER
I20260810 00:55:08.718816 2666 consensus_queue.cc:260] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:08.718997 2664 consensus_queue.cc:260] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:08.719892 2447 heartbeater.cc:503] Master 127.10.97.126:42075 was elected leader, sending a full tablet report...
I20260810 00:55:08.720352 2665 raft_consensus.cc:348] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:08.720793 2665 raft_consensus.cc:374] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:55:08.720930 2665 raft_consensus.cc:729] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fefe09513824e63b56e6a319338a3ab, State: Initialized, Role: FOLLOWER
I20260810 00:55:08.721055 2664 ts_tablet_manager.cc:1428] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f: Time spent starting tablet: real 0.013s user 0.007s sys 0.005s
I20260810 00:55:08.721853 2666 ts_tablet_manager.cc:1428] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260810 00:55:08.722052 2665 consensus_queue.cc:260] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:08.724627 2642 heartbeater.cc:503] Master 127.10.97.126:42075 was elected leader, sending a full tablet report...
I20260810 00:55:08.725181 2665 ts_tablet_manager.cc:1428] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260810 00:55:08.726115 2672 raft_consensus.cc:482] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:55:08.726393 2672 raft_consensus.cc:504] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:08.727910 2672 leader_election.cc:288] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729), 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591)
I20260810 00:55:08.735333 2674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:08.728079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:44591 (local address 127.10.97.65:49061)
0810 00:55:08.728714 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:08.728740 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 00:55:08.728861 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:55:08.729215 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:55:08.729221 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:55:08.729251 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:55:08.729648 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:08.729660 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.730842 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.730846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:08.733041 (+ 2195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:08.733056 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.734515 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.734522 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:08.734699 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:08.734727 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 00:55:08.734808 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 00:55:08.734932 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":71,"thread_start_us":298,"threads_started":1}
I20260810 00:55:08.735621 2662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:08.728904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:48165 (local address 127.10.97.67:37729)
0810 00:55:08.729085 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:08.729090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:08.729485 (+ 395us) server_negotiation.cc:407] Connection header received
0810 00:55:08.729639 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:08.729642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:08.729692 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:08.729777 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:55:08.730235 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.731118 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.733448 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.734609 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.734752 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:08.734890 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:08.735318 (+ 428us) server_negotiation.cc:299] Negotiation successful
0810 00:55:08.735446 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 00:55:08.735638 2663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:08.728754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:49061 (local address 127.10.97.66:44591)
0810 00:55:08.728926 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:08.728931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:08.728950 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:55:08.729010 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:08.729015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:08.729084 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:08.729171 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:55:08.729778 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.730714 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.733181 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.734280 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.734474 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:08.734546 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:08.735199 (+ 653us) server_negotiation.cc:299] Negotiation successful
0810 00:55:08.735334 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 00:55:08.736286 2681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:08.728239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:37729 (local address 127.10.97.65:48165)
0810 00:55:08.729376 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:08.729396 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:55:08.729521 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:55:08.729798 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:55:08.729802 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:55:08.729821 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:55:08.730100 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:08.730108 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.731243 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.731248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:08.733317 (+ 2069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:08.733327 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.734999 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.735003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:08.735134 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:08.735165 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 00:55:08.735839 (+ 674us) client_negotiation.cc:240] Negotiation successful
0810 00:55:08.735951 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1006,"mutex_wait_us":39,"thread_start_us":949,"threads_started":1}
I20260810 00:55:08.736516 2382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ae78a83ded4ff9933051c5d9a25c16" candidate_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3157f17735584b71a3eea3557e73df11" is_pre_election: true
I20260810 00:55:08.736960 2382 raft_consensus.cc:2454] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a65d29f653b84fa7a7dbb52e69b3f19f in term 0.
I20260810 00:55:08.737174 2577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ae78a83ded4ff9933051c5d9a25c16" candidate_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fefe09513824e63b56e6a319338a3ab" is_pre_election: true
I20260810 00:55:08.737566 2072 leader_election.cc:302] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3157f17735584b71a3eea3557e73df11, a65d29f653b84fa7a7dbb52e69b3f19f; no voters:
I20260810 00:55:08.737632 2577 raft_consensus.cc:2454] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a65d29f653b84fa7a7dbb52e69b3f19f in term 0.
I20260810 00:55:08.737890 2672 raft_consensus.cc:2781] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:55:08.738029 2672 raft_consensus.cc:482] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:55:08.738142 2672 raft_consensus.cc:3037] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:55:08.740285 2672 raft_consensus.cc:504] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:08.740811 2672 leader_election.cc:288] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 1 election: Requested vote from peers 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729), 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591)
I20260810 00:55:08.741250 2577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ae78a83ded4ff9933051c5d9a25c16" candidate_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fefe09513824e63b56e6a319338a3ab"
I20260810 00:55:08.741286 2382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ae78a83ded4ff9933051c5d9a25c16" candidate_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3157f17735584b71a3eea3557e73df11"
I20260810 00:55:08.741446 2577 raft_consensus.cc:3037] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:55:08.741446 2382 raft_consensus.cc:3037] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:55:08.744479 2577 raft_consensus.cc:2454] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a65d29f653b84fa7a7dbb52e69b3f19f in term 1.
I20260810 00:55:08.744479 2382 raft_consensus.cc:2454] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a65d29f653b84fa7a7dbb52e69b3f19f in term 1.
I20260810 00:55:08.744864 2072 leader_election.cc:302] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3157f17735584b71a3eea3557e73df11, a65d29f653b84fa7a7dbb52e69b3f19f; no voters:
I20260810 00:55:08.745139 2672 raft_consensus.cc:2781] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:55:08.745540 2672 raft_consensus.cc:686] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 LEADER]: Becoming Leader. State: Replica: a65d29f653b84fa7a7dbb52e69b3f19f, State: Running, Role: LEADER
I20260810 00:55:08.745929 2672 consensus_queue.cc:237] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:08.750087 1958 catalog_manager.cc:5495] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f reported cstate change: term changed from 0 to 1, leader changed from <none> to a65d29f653b84fa7a7dbb52e69b3f19f (127.10.97.65). New cstate: current_term: 1 leader_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } health_report { overall_health: UNKNOWN } } }
I20260810 00:55:08.780766 2660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:08.775896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53436 (local address 127.10.97.65:34371)
0810 00:55:08.776085 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:08.776091 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:08.776170 (+ 79us) server_negotiation.cc:407] Connection header received
0810 00:55:08.776376 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:08.776379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:08.776445 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:08.776587 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:55:08.776929 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.777596 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.778971 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.779379 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.779462 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:08.779894 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:55:08.780339 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:55:08.780445 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:08.780484 (+ 39us) server_negotiation.cc:299] Negotiation successful
0810 00:55:08.780545 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 00:55:08.788098 2577 raft_consensus.cc:1261] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab [term 1 FOLLOWER]: Refusing update from remote peer a65d29f653b84fa7a7dbb52e69b3f19f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:55:08.788093 2382 raft_consensus.cc:1261] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11 [term 1 FOLLOWER]: Refusing update from remote peer a65d29f653b84fa7a7dbb52e69b3f19f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:55:08.789070 2682 consensus_queue.cc:1035] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:55:08.789389 2672 consensus_queue.cc:1035] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:55:08.800019 2685 mvcc.cc:205] Tried to move back new op lower bound from 7316780272786714624 to 7316780272626503680. Current Snapshot: MvccSnapshot[applied={T|T < 7316780272786714624}]
I20260810 00:55:08.812420 1958 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:55:08.812698 1958 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:55:08.816201 1958 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e: Sending DeleteTablet for 3 replicas of tablet 98ae78a83ded4ff9933051c5d9a25c16
I20260810 00:55:08.817204 2149 tablet_service.cc:1514] Processing DeleteTablet for tablet 98ae78a83ded4ff9933051c5d9a25c16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:55:08 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:53420
I20260810 00:55:08.817209 2546 tablet_service.cc:1514] Processing DeleteTablet for tablet 98ae78a83ded4ff9933051c5d9a25c16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:55:08 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:44396
I20260810 00:55:08.817209 2349 tablet_service.cc:1514] Processing DeleteTablet for tablet 98ae78a83ded4ff9933051c5d9a25c16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:55:08 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:46588
I20260810 00:55:08.818463 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 2043
I20260810 00:55:08.848453 2697 tablet_replica.cc:330] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11: stopping tablet replica
I20260810 00:55:08.848775 2699 tablet_replica.cc:330] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f: stopping tablet replica
I20260810 00:55:08.849259 2699 raft_consensus.cc:2229] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:55:08.849252 2697 raft_consensus.cc:2229] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:55:08.849978 2697 raft_consensus.cc:2258] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:55:08.849982 2699 raft_consensus.cc:2258] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:55:08.851203 2698 tablet_replica.cc:330] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab: stopping tablet replica
I20260810 00:55:08.851604 2698 raft_consensus.cc:2229] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:55:08.852113 2698 raft_consensus.cc:2258] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:55:08.853567 2697 ts_tablet_manager.cc:1905] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:55:08.853565 2699 ts_tablet_manager.cc:1905] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 00:55:08 dist-test-slave-jlzn krb5kdc[1886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:55:08.857599 2698 ts_tablet_manager.cc:1905] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:55:08.861630 2699 ts_tablet_manager.cc:1918] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:55:08.861953 2697 ts_tablet_manager.cc:1918] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:55:08.862176 2697 log.cc:1196] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/98ae78a83ded4ff9933051c5d9a25c16
I20260810 00:55:08.862761 2697 ts_tablet_manager.cc:1939] T 98ae78a83ded4ff9933051c5d9a25c16 P 3157f17735584b71a3eea3557e73df11: Deleting consensus metadata
I20260810 00:55:08.863380 2699 log.cc:1196] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/98ae78a83ded4ff9933051c5d9a25c16
I20260810 00:55:08.864723 1936 catalog_manager.cc:4841] TS 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591): tablet 98ae78a83ded4ff9933051c5d9a25c16 (table test-table [id=33bbe251fa8e457fb550e68a7fd79efe]) successfully deleted
I20260810 00:55:08.865489 2660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:08.819710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53438 (local address 127.10.97.65:34371)
0810 00:55:08.819906 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:08.819912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:08.841024 (+ 21112us) server_negotiation.cc:407] Connection header received
0810 00:55:08.841548 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:08.841572 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:08.841678 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:08.841758 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:08.843950 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.845168 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.850394 (+ 5226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.851121 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.851255 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:08.857277 (+ 6022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:08.857292 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:08.857310 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:08.857364 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:08.861687 (+ 4323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:08.862540 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:08.862549 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:08.862564 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:08.862853 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:08.863383 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:08.863388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:08.863391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:08.863879 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:08.864160 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:08.864806 (+ 646us) server_negotiation.cc:299] Negotiation successful
0810 00:55:08.865035 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 00:55:08.865221 2699 ts_tablet_manager.cc:1939] T 98ae78a83ded4ff9933051c5d9a25c16 P a65d29f653b84fa7a7dbb52e69b3f19f: Deleting consensus metadata
I20260810 00:55:08.866605 2242 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:55:08.867525 2698 ts_tablet_manager.cc:1918] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:55:08.867689 2698 log.cc:1196] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/98ae78a83ded4ff9933051c5d9a25c16
I20260810 00:55:08.868253 2698 ts_tablet_manager.cc:1939] T 98ae78a83ded4ff9933051c5d9a25c16 P 5fefe09513824e63b56e6a319338a3ab: Deleting consensus metadata
I20260810 00:55:08.869959 1935 catalog_manager.cc:4841] TS 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729): tablet 98ae78a83ded4ff9933051c5d9a25c16 (table test-table [id=33bbe251fa8e457fb550e68a7fd79efe]) successfully deleted
I20260810 00:55:08.967530 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 2043
I20260810 00:55:08.991338 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 2258
W20260810 00:55:08.991261 1934 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.97.65:34371 (error 108)
I20260810 00:55:08.993175 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:08.992744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34371 (local address 127.0.0.1:53448)
0810 00:55:08.992911 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:08.993023 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 00:55:08.993374 1934 catalog_manager.cc:4584] TS a65d29f653b84fa7a7dbb52e69b3f19f (127.10.97.65:34371): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 98ae78a83ded4ff9933051c5d9a25c16: Network error: Client connection negotiation failed: client connection to 127.10.97.65:34371: connect: Connection refused (error 111)
Aug 10 00:55:08 dist-test-slave-jlzn krb5kdc[1886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.66@KRBTEST.COM
I20260810 00:55:09.006260 2663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:08.992107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46592 (local address 127.10.97.66:44591)
0810 00:55:08.992297 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:08.992303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:08.992445 (+ 142us) server_negotiation.cc:407] Connection header received
0810 00:55:08.992626 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:08.992631 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:08.992700 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:08.992798 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:08.994238 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.994983 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.996512 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:08.996891 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:08.997001 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:09.000972 (+ 3971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:09.000984 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:09.000999 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:09.001038 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:09.003501 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:09.004281 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:09.004288 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:09.004290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:09.004418 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:09.004791 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:09.004795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:09.004797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:09.005161 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:09.005459 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:09.005760 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 00:55:09.005959 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 00:55:09.006830 2437 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:55:09.046815 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:09.046392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34371 (local address 127.0.0.1:53460)
0810 00:55:09.046580 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:09.046669 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 00:55:09.047758 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:09.047443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34371 (local address 127.0.0.1:53476)
0810 00:55:09.047570 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:09.047652 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 00:55:09.093158 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 2258
I20260810 00:55:09.108286 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:09.107703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34371 (local address 127.0.0.1:53492)
0810 00:55:09.107963 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:09.108084 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 00:55:09.109416 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:09.109010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34371 (local address 127.0.0.1:53504)
0810 00:55:09.109173 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:09.109263 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 00:55:09.114231 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 2452
Aug 10 00:55:09 dist-test-slave-jlzn krb5kdc[1886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.67@KRBTEST.COM
I20260810 00:55:09.128530 2662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:09.114948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44402 (local address 127.10.97.67:37729)
0810 00:55:09.115116 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:09.115120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:09.115149 (+ 29us) server_negotiation.cc:407] Connection header received
0810 00:55:09.115217 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:09.115221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:09.115284 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:09.115356 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:09.116679 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:09.117404 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:09.118618 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:09.119110 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:09.119217 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:09.123373 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:09.123387 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:09.123405 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:09.123448 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:09.126141 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:09.126799 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:09.126806 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:09.126808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:09.126943 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:09.127213 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:09.127215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:09.127217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:09.127556 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:09.127774 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:09.128067 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 00:55:09.128241 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 00:55:09.129117 2634 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:55:09.189922 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:09.189516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34371 (local address 127.0.0.1:53512)
0810 00:55:09.189693 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:09.189781 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 00:55:09.190881 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:09.190529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34371 (local address 127.0.0.1:53528)
0810 00:55:09.190697 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:09.190765 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260810 00:55:09.215601 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 2452
I20260810 00:55:09.239943 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 1906
I20260810 00:55:09.241034 2004 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:55:09.348368 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 1906
I20260810 00:55:29.375295 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:42075
--webserver_interface=127.10.97.126
--webserver_port=34067
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:39661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:42075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:55:29.708961 2711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:55:29.709214 2711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:55:29.709336 2711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:55:29.716176 2711 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 00:55:29.716259 2711 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 00:55:29.716320 2711 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 00:55:29.716388 2711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:55:29.716459 2711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:55:29.716550 2711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:55:29.716612 2711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:55:29.716693 2711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:55:29.725831 2711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:42075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:42075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=34067
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:55:29.726882 2711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:55:29.728132 2711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:55:29.751224 2718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:29.751190 2724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:29.751201 2721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:55:29.751778 2711 server_base.cc:1034] running on GCE node
I20260810 00:55:29.752861 2711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:55:29.754293 2711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:55:29.755515 2711 hybrid_clock.cc:648] HybridClock initialized: now 1786323329755441 us; error 44 us; skew 500 ppm
I20260810 00:55:29.755923 2711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:55:29 dist-test-slave-jlzn krb5kdc[1886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323329, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:55:29.759479 2711 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:55:29.761045 2711 webserver.cc:460] Webserver started at http://127.10.97.126:34067/ using document root <none> and password file <none>
I20260810 00:55:29.761781 2711 fs_manager.cc:359] Metadata directory not provided
I20260810 00:55:29.761888 2711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:55:29.767568 2711 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 00:55:29.770399 2732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:29.771697 2711 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 00:55:29.771888 2711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "8cc21197f71449078605f8078a16825e"
format_stamp: "Formatted at 2026-08-10 00:55:07 on dist-test-slave-jlzn"
server_key: "2221b4e1457a6a13623b621ffe498eb9"
server_key_iv: "197294f61c048fd25c902d27aa5528ea"
server_key_version: "encryptionkey@0"
I20260810 00:55:29.772884 2711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:55:29.812564 2711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:55:29.816871 2711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:55:29.817160 2711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:55:29.835528 2711 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:42075
I20260810 00:55:29.835577 2806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:42075 every 8 connection(s)
I20260810 00:55:29.837105 2711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:55:29.842767 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 2711
I20260810 00:55:29.843824 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:34371
--local_ip_for_outbound_sockets=127.10.97.65
--tserver_master_addrs=127.10.97.126:42075
--webserver_port=46255
--webserver_interface=127.10.97.65
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:39661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:55:29.844470 2811 sys_catalog.cc:263] Verifying existing consensus state
I20260810 00:55:29.848798 2811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e: Bootstrap starting.
I20260810 00:55:29.869457 2811 log.cc:824] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e: Log is configured to *not* fsync() on all Append() calls
I20260810 00:55:29.881191 2811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 00:55:29.881685 2811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e: Bootstrap complete.
I20260810 00:55:29.890326 2811 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc21197f71449078605f8078a16825e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42075 } }
I20260810 00:55:29.891116 2811 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cc21197f71449078605f8078a16825e, State: Initialized, Role: FOLLOWER
I20260810 00:55:29.891706 2811 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc21197f71449078605f8078a16825e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42075 } }
I20260810 00:55:29.891952 2811 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:55:29.892069 2811 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:55:29.892232 2811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:55:29.895781 2811 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc21197f71449078605f8078a16825e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42075 } }
I20260810 00:55:29.896297 2811 leader_election.cc:302] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cc21197f71449078605f8078a16825e; no voters:
I20260810 00:55:29.896905 2811 leader_election.cc:288] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 00:55:29.897032 2817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 2 FOLLOWER]: Leader election won for term 2
I20260810 00:55:29.898193 2817 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 2 LEADER]: Becoming Leader. State: Replica: 8cc21197f71449078605f8078a16825e, State: Running, Role: LEADER
I20260810 00:55:29.898734 2817 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc21197f71449078605f8078a16825e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42075 } }
I20260810 00:55:29.899082 2811 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:55:29.901259 2819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cc21197f71449078605f8078a16825e. Latest consensus state: current_term: 2 leader_uuid: "8cc21197f71449078605f8078a16825e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc21197f71449078605f8078a16825e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42075 } } }
I20260810 00:55:29.901283 2818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8cc21197f71449078605f8078a16825e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc21197f71449078605f8078a16825e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42075 } } }
I20260810 00:55:29.901665 2819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [sys.catalog]: This master's current role is: LEADER
I20260810 00:55:29.901669 2818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [sys.catalog]: This master's current role is: LEADER
I20260810 00:55:29.913445 2825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:55:29.921185 2825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:55:29.922575 2825 catalog_manager.cc:1259] Loaded cluster ID: b3a7e13690c849189952149cd0196486
I20260810 00:55:29.922736 2825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:55:29.926733 2825 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:55:29.928277 2825 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e: Loaded TSK: 0
I20260810 00:55:29.969506 2825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e: Generated new TSK 1
I20260810 00:55:29.969926 2825 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 00:55:30.191241 2814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:55:30.191524 2814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:55:30.191660 2814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:55:30.198616 2814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:55:30.198727 2814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:55:30.198873 2814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:55:30.208206 2814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:34371
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=46255
--tserver_master_addrs=127.10.97.126:42075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:55:30.209317 2814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:55:30.210603 2814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:55:30.224692 2853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:30.224766 2849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:30.225323 2848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:55:30.226050 2814 server_base.cc:1034] running on GCE node
I20260810 00:55:30.226644 2814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:55:30.227526 2814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:55:30.228734 2814 hybrid_clock.cc:648] HybridClock initialized: now 1786323330228689 us; error 40 us; skew 500 ppm
I20260810 00:55:30.229008 2814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:55:30 dist-test-slave-jlzn krb5kdc[1886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323330, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:55:30.234511 2814 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:55:30.235944 2814 webserver.cc:460] Webserver started at http://127.10.97.65:46255/ using document root <none> and password file <none>
I20260810 00:55:30.236553 2814 fs_manager.cc:359] Metadata directory not provided
I20260810 00:55:30.236677 2814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:55:30.241652 2814 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 00:55:30.244166 2862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:30.245412 2814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 00:55:30.245561 2814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "a65d29f653b84fa7a7dbb52e69b3f19f"
format_stamp: "Formatted at 2026-08-10 00:55:07 on dist-test-slave-jlzn"
server_key: "b1b4613a97c41d82fe0b450fc5704b0a"
server_key_iv: "746e341fbb51fc81a6285609de39174e"
server_key_version: "encryptionkey@0"
I20260810 00:55:30.246455 2814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:55:30.265785 2814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:55:30.269652 2814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:55:30.269884 2814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:55:30.270763 2814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:55:30.272089 2814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:55:30.272176 2814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:30.272285 2814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:55:30.272359 2814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:30.301481 2814 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:34371
I20260810 00:55:30.301568 3038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:34371 every 8 connection(s)
I20260810 00:55:30.302774 2814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:55:30.312417 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 2814
I20260810 00:55:30.313028 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:44591
--local_ip_for_outbound_sockets=127.10.97.66
--tserver_master_addrs=127.10.97.126:42075
--webserver_port=44847
--webserver_interface=127.10.97.66
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:39661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 00:55:30 dist-test-slave-jlzn krb5kdc[1886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323330, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:55:30.324158 3042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:30.305329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:37029 (local address 127.10.97.126:42075)
0810 00:55:30.305839 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:30.305851 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:30.306528 (+ 677us) server_negotiation.cc:407] Connection header received
0810 00:55:30.308597 (+ 2069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:30.308610 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:30.308929 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:30.309197 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:30.310489 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:30.311709 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:30.313277 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:30.313900 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:30.314084 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:30.317238 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:30.317246 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:30.317258 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:30.317292 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:30.320111 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:30.320938 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:30.320944 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:30.320950 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:30.321105 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:30.321540 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:30.321543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:30.321545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:30.321970 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:30.322113 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:30.322851 (+ 738us) server_negotiation.cc:299] Negotiation successful
0810 00:55:30.323155 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":218,"threads_started":1}
I20260810 00:55:30.324222 3043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:30.305569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:37029)
0810 00:55:30.306370 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:30.306423 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 00:55:30.308137 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:55:30.309312 (+ 1175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:55:30.309319 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:55:30.309801 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:55:30.310309 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:30.310326 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:30.311857 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:30.311861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:30.313061 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:30.313074 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:30.314131 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:30.314140 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:30.314373 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:30.314958 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:55:30.314984 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:55:30.316986 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:55:30.320280 (+ 3294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:55:30.320288 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:55:30.320300 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:55:30.320807 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:55:30.321219 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:55:30.321223 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:55:30.321227 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:55:30.321445 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:55:30.322091 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:55:30.322097 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:55:30.322444 (+ 347us) client_negotiation.cc:769] Sending connection context
0810 00:55:30.322701 (+ 257us) client_negotiation.cc:240] Negotiation successful
0810 00:55:30.323122 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":322,"threads_started":1}
I20260810 00:55:30.327109 3040 heartbeater.cc:347] Connected to a master server at 127.10.97.126:42075
I20260810 00:55:30.327368 3040 heartbeater.cc:464] Registering TS with master...
I20260810 00:55:30.328054 3040 heartbeater.cc:511] Master 127.10.97.126:42075 requested a full tablet report, sending...
I20260810 00:55:30.330335 2759 ts_manager.cc:194] Registered new tserver with Master: a65d29f653b84fa7a7dbb52e69b3f19f (127.10.97.65:34371)
I20260810 00:55:30.332314 2759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:37029
W20260810 00:55:30.636977 3045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:55:30.637223 3045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:55:30.637317 3045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:55:30.644508 3045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:55:30.644601 3045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:55:30.644712 3045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:55:30.653947 3045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:44591
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=44847
--tserver_master_addrs=127.10.97.126:42075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:55:30.654981 3045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:55:30.656173 3045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:55:30.668258 3057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:30.668313 3055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:30.668332 3060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:55:30.668967 3045 server_base.cc:1034] running on GCE node
I20260810 00:55:30.669525 3045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:55:30.670380 3045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:55:30.671573 3045 hybrid_clock.cc:648] HybridClock initialized: now 1786323330671527 us; error 39 us; skew 500 ppm
I20260810 00:55:30.671902 3045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:55:30 dist-test-slave-jlzn krb5kdc[1886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323330, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:55:30.674796 3045 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:55:30.676098 3045 webserver.cc:460] Webserver started at http://127.10.97.66:44847/ using document root <none> and password file <none>
I20260810 00:55:30.676697 3045 fs_manager.cc:359] Metadata directory not provided
I20260810 00:55:30.676827 3045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:55:30.681672 3045 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260810 00:55:30.684147 3071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:30.685326 3045 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:55:30.685475 3045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "3157f17735584b71a3eea3557e73df11"
format_stamp: "Formatted at 2026-08-10 00:55:08 on dist-test-slave-jlzn"
server_key: "338825120f9c5a5043fba4bd0167b527"
server_key_iv: "325b92fd55f7e903ed4f64d6a4fd34b4"
server_key_version: "encryptionkey@0"
I20260810 00:55:30.686374 3045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:55:30.698501 3045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:55:30.701973 3045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:55:30.702226 3045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:55:30.703087 3045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:55:30.704325 3045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:55:30.704422 3045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:30.704526 3045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:55:30.704604 3045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:30.733884 3045 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:44591
I20260810 00:55:30.733942 3242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:44591 every 8 connection(s)
I20260810 00:55:30.735245 3045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:55:30.741492 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 3045
I20260810 00:55:30.742241 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:37729
--local_ip_for_outbound_sockets=127.10.97.67
--tserver_master_addrs=127.10.97.126:42075
--webserver_port=46395
--webserver_interface=127.10.97.67
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:39661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 00:55:30 dist-test-slave-jlzn krb5kdc[1886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323330, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:55:30.754989 3042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:30.737613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:39839 (local address 127.10.97.126:42075)
0810 00:55:30.737791 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:30.737796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:30.738843 (+ 1047us) server_negotiation.cc:407] Connection header received
0810 00:55:30.740567 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:30.740571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:30.740648 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:30.740732 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:30.742514 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:30.743214 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:30.745312 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:30.745755 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:30.745847 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:30.749672 (+ 3825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:30.749678 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:30.749681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:30.749714 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:30.752098 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:30.752841 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:30.752845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:30.752847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:30.752958 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:30.753345 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:30.753348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:30.753350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:30.753593 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:30.753702 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:30.754451 (+ 749us) server_negotiation.cc:299] Negotiation successful
0810 00:55:30.754678 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 00:55:30.755860 3245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:30.738017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:39839)
0810 00:55:30.738689 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:30.738728 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 00:55:30.740335 (+ 1607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:55:30.740909 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:55:30.740917 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:55:30.741605 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:55:30.742340 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:30.742366 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:30.743372 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:30.743377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:30.745167 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:30.745190 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:30.745957 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:30.745963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:30.746227 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:30.746919 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:55:30.746945 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:55:30.749443 (+ 2498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:55:30.752223 (+ 2780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:55:30.752231 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:55:30.752244 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:55:30.752747 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:55:30.753044 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:55:30.753048 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:55:30.753051 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:55:30.753258 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:55:30.753718 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:55:30.753725 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:55:30.754075 (+ 350us) client_negotiation.cc:769] Sending connection context
0810 00:55:30.754319 (+ 244us) client_negotiation.cc:240] Negotiation successful
0810 00:55:30.754778 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":255,"threads_started":1}
I20260810 00:55:30.757144 3243 heartbeater.cc:347] Connected to a master server at 127.10.97.126:42075
I20260810 00:55:30.757436 3243 heartbeater.cc:464] Registering TS with master...
I20260810 00:55:30.758201 3243 heartbeater.cc:511] Master 127.10.97.126:42075 requested a full tablet report, sending...
I20260810 00:55:30.759621 2759 ts_manager.cc:194] Registered new tserver with Master: 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591)
I20260810 00:55:30.760432 2759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:39839
W20260810 00:55:31.077337 3246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:55:31.077608 3246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:55:31.077759 3246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:55:31.084659 3246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:55:31.084753 3246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:55:31.084898 3246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:55:31.094120 3246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:37729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=46395
--tserver_master_addrs=127.10.97.126:42075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:55:31.095237 3246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:55:31.096489 3246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:55:31.108773 3255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:31.108804 3258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:31.108728 3256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:55:31.109732 3246 server_base.cc:1034] running on GCE node
I20260810 00:55:31.110321 3246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:55:31.111212 3246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:55:31.112416 3246 hybrid_clock.cc:648] HybridClock initialized: now 1786323331112374 us; error 32 us; skew 500 ppm
I20260810 00:55:31.112715 3246 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:55:31 dist-test-slave-jlzn krb5kdc[1886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323331, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:55:31.115599 3246 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:55:31.116984 3246 webserver.cc:460] Webserver started at http://127.10.97.67:46395/ using document root <none> and password file <none>
I20260810 00:55:31.117583 3246 fs_manager.cc:359] Metadata directory not provided
I20260810 00:55:31.117697 3246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:55:31.122671 3246 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 00:55:31.125208 3270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:31.126386 3246 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:55:31.126523 3246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "5fefe09513824e63b56e6a319338a3ab"
format_stamp: "Formatted at 2026-08-10 00:55:08 on dist-test-slave-jlzn"
server_key: "80d18bd3b04e11902af9621ab7186443"
server_key_iv: "47a67ddd023a7ac190e2783ae6bfe122"
server_key_version: "encryptionkey@0"
I20260810 00:55:31.127401 3246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:55:31.159888 3246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:55:31.163503 3246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:55:31.163729 3246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:55:31.164660 3246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:55:31.165889 3246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:55:31.165977 3246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:31.166057 3246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:55:31.166109 3246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:31.195956 3246 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:37729
I20260810 00:55:31.196022 3435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:37729 every 8 connection(s)
I20260810 00:55:31.197355 3246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:55:31.201143 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 3246
Aug 10 00:55:31 dist-test-slave-jlzn krb5kdc[1886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323331, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:55:31.217731 3439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.202123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51898 (local address 127.10.97.126:42075)
0810 00:55:31.202546 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:31.202551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:31.204329 (+ 1778us) server_negotiation.cc:407] Connection header received
0810 00:55:31.204453 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:31.204458 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:31.204524 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:31.204633 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:31.206101 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.207170 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.208609 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.209166 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.209275 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:31.212183 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:31.212189 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:31.212192 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:31.212243 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:31.215049 (+ 2806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:31.215766 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:31.215770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:31.215772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:31.215923 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:31.216286 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:31.216291 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:31.216295 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:31.216546 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:31.216766 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:31.217103 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 00:55:31.217325 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":222,"threads_started":1}
I20260810 00:55:31.217840 3042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.199675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:35557 (local address 127.10.97.126:42075)
0810 00:55:31.199901 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:31.199907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:31.201168 (+ 1261us) server_negotiation.cc:407] Connection header received
0810 00:55:31.203273 (+ 2105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:31.203278 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:31.203357 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:31.203442 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:31.205047 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.205799 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.207769 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.208217 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.208324 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:31.211932 (+ 3608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:31.211937 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:31.211940 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:31.211971 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:31.214994 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:31.215624 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:31.215629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:31.215631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:31.215738 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:31.216164 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:31.216168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:31.216170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:31.216416 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:31.216541 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:31.217167 (+ 626us) server_negotiation.cc:299] Negotiation successful
0810 00:55:31.217476 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":94,"server-negotiator.queue_time_us":62}
I20260810 00:55:31.218616 3438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.200217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:35557)
0810 00:55:31.200990 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:31.201046 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 00:55:31.203021 (+ 1975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:55:31.203693 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:55:31.203700 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:55:31.204209 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:55:31.204898 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:31.204918 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.205958 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.205961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:31.207644 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:31.207656 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.208427 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.208434 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:31.208700 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:31.209438 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:55:31.209462 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:55:31.211728 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:55:31.215113 (+ 3385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:55:31.215122 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:55:31.215135 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:55:31.215507 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:55:31.215858 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:55:31.215863 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:55:31.215866 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:55:31.216042 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:55:31.216537 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:55:31.216544 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:55:31.216830 (+ 286us) client_negotiation.cc:769] Sending connection context
0810 00:55:31.217044 (+ 214us) client_negotiation.cc:240] Negotiation successful
0810 00:55:31.217482 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":268,"threads_started":1}
I20260810 00:55:31.219938 3436 heartbeater.cc:347] Connected to a master server at 127.10.97.126:42075
I20260810 00:55:31.220229 3436 heartbeater.cc:464] Registering TS with master...
I20260810 00:55:31.220868 3436 heartbeater.cc:511] Master 127.10.97.126:42075 requested a full tablet report, sending...
I20260810 00:55:31.222149 2759 ts_manager.cc:194] Registered new tserver with Master: 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729)
I20260810 00:55:31.222915 2759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:35557
I20260810 00:55:31.232016 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:55:31.238224 3439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.232885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51906 (local address 127.10.97.126:42075)
0810 00:55:31.233050 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:31.233055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:31.233396 (+ 341us) server_negotiation.cc:407] Connection header received
0810 00:55:31.233613 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:31.233617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:31.233739 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:31.233853 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:55:31.234237 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.234938 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.236402 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.236824 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.236928 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:31.237392 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:55:31.237641 (+ 249us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 00:55:31.237969 (+ 328us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51906: token expired
Metrics: {"server-negotiator.queue_time_us":42}
W20260810 00:55:31.238328 3439 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51906: token expired
W20260810 00:55:31.238796 2649 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.97.126:42075)
I20260810 00:55:31.249997 3439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.239247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51922 (local address 127.10.97.126:42075)
0810 00:55:31.239386 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:31.239389 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:31.239404 (+ 15us) server_negotiation.cc:407] Connection header received
0810 00:55:31.239504 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:31.239508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:31.239554 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:31.239621 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:31.240892 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.241564 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.242810 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.243207 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.243287 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:31.245654 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:31.245659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:31.245662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:31.245688 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:31.247937 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:31.248519 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:31.248523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:31.248525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:31.248644 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:31.248898 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:31.248901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:31.248903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:31.249162 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:31.249370 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:31.249614 (+ 244us) server_negotiation.cc:299] Negotiation successful
0810 00:55:31.249758 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 00:55:31.252990 2759 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:55:31.259939 2759 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:55:31.290894 3454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.270469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:37729 (local address 127.0.0.1:39396)
0810 00:55:31.283172 (+ 12703us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:31.283192 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 00:55:31.283368 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:55:31.283967 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:55:31.283971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:55:31.283996 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:55:31.285024 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:31.285038 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.286426 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.286431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:31.288489 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:31.288501 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.290174 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.290181 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:31.290334 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:31.290365 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 00:55:31.290439 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 00:55:31.290547 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12564,"mutex_wait_us":65,"spinlock_wait_cycles":896,"thread_start_us":10337,"threads_started":1}
I20260810 00:55:31.291232 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.270805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39396 (local address 127.10.97.67:37729)
0810 00:55:31.282758 (+ 11953us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:31.282766 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:31.283436 (+ 670us) server_negotiation.cc:407] Connection header received
0810 00:55:31.283534 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:31.283541 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:31.283766 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:31.284221 (+ 455us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:55:31.285207 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.286296 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.288622 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.289732 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.289867 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:31.290072 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:31.290508 (+ 436us) server_negotiation.cc:299] Negotiation successful
0810 00:55:31.290666 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11799,"thread_start_us":11638,"threads_started":1}
I20260810 00:55:31.291347 3450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.270377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50172 (local address 127.10.97.65:34371)
0810 00:55:31.282372 (+ 11995us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:31.282380 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:31.282409 (+ 29us) server_negotiation.cc:407] Connection header received
0810 00:55:31.282502 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:31.282510 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:31.282749 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:31.282925 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:55:31.283901 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.285349 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.288320 (+ 2971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.289678 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.289813 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:31.290011 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:31.290895 (+ 884us) server_negotiation.cc:299] Negotiation successful
0810 00:55:31.291050 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11821,"thread_start_us":11693,"threads_started":1}
I20260810 00:55:31.291608 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.270187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34371 (local address 127.0.0.1:50172)
0810 00:55:31.280431 (+ 10244us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:31.280482 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 00:55:31.280645 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:55:31.283194 (+ 2549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:55:31.283202 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:55:31.283238 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:55:31.283706 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:31.283720 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.285480 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.285486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:31.287778 (+ 2292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:31.287794 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.289880 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.289886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:31.290072 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:31.290129 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 00:55:31.290258 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 00:55:31.290419 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10007,"mutex_wait_us":1,"spinlock_wait_cycles":24960,"thread_start_us":12980,"threads_started":1}
I20260810 00:55:31.292155 3448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.271709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:44591 (local address 127.0.0.1:42440)
0810 00:55:31.283436 (+ 11727us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:31.283450 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 00:55:31.283557 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:55:31.284320 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:55:31.284324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:55:31.284349 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:55:31.284728 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:31.284741 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.286507 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.286513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:31.289205 (+ 2692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:31.289221 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.291343 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.291351 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:31.291525 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:31.291562 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 00:55:31.291671 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 00:55:31.291788 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11611,"mutex_wait_us":20,"thread_start_us":11217,"threads_started":1}
I20260810 00:55:31.292460 3455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.272300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42440 (local address 127.10.97.66:44591)
0810 00:55:31.283006 (+ 10706us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:31.283013 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:31.283531 (+ 518us) server_negotiation.cc:407] Connection header received
0810 00:55:31.283693 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:31.283698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:31.283903 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:31.284086 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:55:31.284972 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.286334 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.289428 (+ 3094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.290834 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.290962 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:31.291135 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:31.291867 (+ 732us) server_negotiation.cc:299] Negotiation successful
0810 00:55:31.292034 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10553,"thread_start_us":10456,"threads_started":1}
I20260810 00:55:31.295126 3340 tablet_service.cc:1467] Processing CreateTablet for tablet 408b1aabf266427cb4e59b3cbc0fa92c (DEFAULT_TABLE table=test-table [id=6caa5ad8cae644988e17d12f84228908]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:55:31.295135 2933 tablet_service.cc:1467] Processing CreateTablet for tablet 408b1aabf266427cb4e59b3cbc0fa92c (DEFAULT_TABLE table=test-table [id=6caa5ad8cae644988e17d12f84228908]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:55:31.295567 3150 tablet_service.cc:1467] Processing CreateTablet for tablet 408b1aabf266427cb4e59b3cbc0fa92c (DEFAULT_TABLE table=test-table [id=6caa5ad8cae644988e17d12f84228908]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:55:31.296607 2933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 408b1aabf266427cb4e59b3cbc0fa92c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:55:31.296607 3340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 408b1aabf266427cb4e59b3cbc0fa92c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:55:31.296720 3150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 408b1aabf266427cb4e59b3cbc0fa92c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:55:31.312604 3462 tablet_bootstrap.cc:492] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab: Bootstrap starting.
I20260810 00:55:31.314569 3460 tablet_bootstrap.cc:492] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f: Bootstrap starting.
I20260810 00:55:31.316479 3462 tablet_bootstrap.cc:654] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab: Neither blocks nor log segments found. Creating new log.
I20260810 00:55:31.317356 3460 tablet_bootstrap.cc:654] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f: Neither blocks nor log segments found. Creating new log.
I20260810 00:55:31.317812 3462 log.cc:824] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab: Log is configured to *not* fsync() on all Append() calls
I20260810 00:55:31.318280 3460 log.cc:824] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f: Log is configured to *not* fsync() on all Append() calls
I20260810 00:55:31.321146 3462 tablet_bootstrap.cc:492] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab: No bootstrap required, opened a new log
I20260810 00:55:31.321430 3462 ts_tablet_manager.cc:1397] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 00:55:31.324687 3457 tablet_bootstrap.cc:492] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11: Bootstrap starting.
I20260810 00:55:31.327250 3460 tablet_bootstrap.cc:492] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f: No bootstrap required, opened a new log
I20260810 00:55:31.327396 3457 tablet_bootstrap.cc:654] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11: Neither blocks nor log segments found. Creating new log.
I20260810 00:55:31.327497 3460 ts_tablet_manager.cc:1397] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.006s
I20260810 00:55:31.328495 3457 log.cc:824] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11: Log is configured to *not* fsync() on all Append() calls
I20260810 00:55:31.331173 3457 tablet_bootstrap.cc:492] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11: No bootstrap required, opened a new log
I20260810 00:55:31.331426 3457 ts_tablet_manager.cc:1397] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260810 00:55:31.331209 3462 raft_consensus.cc:348] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:31.331650 3462 raft_consensus.cc:374] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:55:31.331753 3462 raft_consensus.cc:729] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fefe09513824e63b56e6a319338a3ab, State: Initialized, Role: FOLLOWER
I20260810 00:55:31.332351 3462 consensus_queue.cc:260] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:31.333410 3436 heartbeater.cc:503] Master 127.10.97.126:42075 was elected leader, sending a full tablet report...
I20260810 00:55:31.334044 3462 ts_tablet_manager.cc:1428] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab: Time spent starting tablet: real 0.012s user 0.010s sys 0.004s
I20260810 00:55:31.335340 3040 heartbeater.cc:503] Master 127.10.97.126:42075 was elected leader, sending a full tablet report...
I20260810 00:55:31.337046 3460 raft_consensus.cc:348] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:31.337460 3460 raft_consensus.cc:374] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:55:31.337551 3460 raft_consensus.cc:729] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a65d29f653b84fa7a7dbb52e69b3f19f, State: Initialized, Role: FOLLOWER
I20260810 00:55:31.338183 3460 consensus_queue.cc:260] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:31.339788 3460 ts_tablet_manager.cc:1428] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f: Time spent starting tablet: real 0.012s user 0.007s sys 0.006s
I20260810 00:55:31.340891 3457 raft_consensus.cc:348] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:31.341267 3457 raft_consensus.cc:374] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:55:31.341377 3457 raft_consensus.cc:729] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3157f17735584b71a3eea3557e73df11, State: Initialized, Role: FOLLOWER
I20260810 00:55:31.341931 3457 consensus_queue.cc:260] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:31.342911 3243 heartbeater.cc:503] Master 127.10.97.126:42075 was elected leader, sending a full tablet report...
I20260810 00:55:31.343340 3457 ts_tablet_manager.cc:1428] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260810 00:55:31.359290 3469 raft_consensus.cc:482] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:55:31.359490 3469 raft_consensus.cc:504] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:31.360877 3469 leader_election.cc:288] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a65d29f653b84fa7a7dbb52e69b3f19f (127.10.97.65:34371), 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591)
I20260810 00:55:31.367413 3450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.361190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:35689 (local address 127.10.97.65:34371)
0810 00:55:31.361366 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:31.361371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:31.361470 (+ 99us) server_negotiation.cc:407] Connection header received
0810 00:55:31.361712 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:31.361717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:31.361787 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:31.361868 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:55:31.362330 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.363263 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.365612 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.366693 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.366839 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:31.366909 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:31.367098 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 00:55:31.367226 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 00:55:31.367513 3438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.361085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34371 (local address 127.10.97.67:35689)
0810 00:55:31.361390 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:31.361407 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:55:31.361520 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:55:31.361922 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:55:31.361928 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:55:31.361957 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:55:31.362202 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:31.362212 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.363398 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.363402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:31.365472 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:31.365487 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.366824 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.366829 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:31.366956 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:31.366978 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 00:55:31.367047 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 00:55:31.367165 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":65,"spinlock_wait_cycles":24448}
I20260810 00:55:31.367981 3455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.361369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:59605 (local address 127.10.97.66:44591)
0810 00:55:31.361554 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:31.361560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:31.362007 (+ 447us) server_negotiation.cc:407] Connection header received
0810 00:55:31.362157 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:31.362161 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:31.362210 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:31.362281 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:55:31.362741 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.363584 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.365890 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.367027 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.367165 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:31.367229 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:31.367663 (+ 434us) server_negotiation.cc:299] Negotiation successful
0810 00:55:31.367779 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 00:55:31.368134 3476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.361110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:44591 (local address 127.10.97.67:59605)
0810 00:55:31.361913 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:31.361929 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:55:31.362051 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:55:31.362308 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:55:31.362313 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:55:31.362336 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:55:31.362605 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:31.362612 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.363721 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.363726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:31.365758 (+ 2032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:31.365769 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.367223 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.367228 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:31.367395 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:31.367429 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 00:55:31.367647 (+ 218us) client_negotiation.cc:240] Negotiation successful
0810 00:55:31.367766 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":54,"thread_start_us":524,"threads_started":1}
I20260810 00:55:31.368464 2963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "408b1aabf266427cb4e59b3cbc0fa92c" candidate_uuid: "5fefe09513824e63b56e6a319338a3ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" is_pre_election: true
I20260810 00:55:31.368942 2963 raft_consensus.cc:2454] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fefe09513824e63b56e6a319338a3ab in term 0.
I20260810 00:55:31.368999 3178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "408b1aabf266427cb4e59b3cbc0fa92c" candidate_uuid: "5fefe09513824e63b56e6a319338a3ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3157f17735584b71a3eea3557e73df11" is_pre_election: true
I20260810 00:55:31.369458 3178 raft_consensus.cc:2454] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fefe09513824e63b56e6a319338a3ab in term 0.
I20260810 00:55:31.369558 3272 leader_election.cc:302] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fefe09513824e63b56e6a319338a3ab, a65d29f653b84fa7a7dbb52e69b3f19f; no voters:
I20260810 00:55:31.369917 3469 raft_consensus.cc:2781] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:55:31.370078 3469 raft_consensus.cc:482] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:55:31.370199 3469 raft_consensus.cc:3037] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:55:31.372347 3469 raft_consensus.cc:504] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:31.372876 3469 leader_election.cc:288] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [CANDIDATE]: Term 1 election: Requested vote from peers a65d29f653b84fa7a7dbb52e69b3f19f (127.10.97.65:34371), 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591)
I20260810 00:55:31.373220 2963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "408b1aabf266427cb4e59b3cbc0fa92c" candidate_uuid: "5fefe09513824e63b56e6a319338a3ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f"
I20260810 00:55:31.373281 3178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "408b1aabf266427cb4e59b3cbc0fa92c" candidate_uuid: "5fefe09513824e63b56e6a319338a3ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3157f17735584b71a3eea3557e73df11"
I20260810 00:55:31.373375 2963 raft_consensus.cc:3037] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:55:31.373440 3178 raft_consensus.cc:3037] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:55:31.375388 2963 raft_consensus.cc:2454] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fefe09513824e63b56e6a319338a3ab in term 1.
I20260810 00:55:31.375500 3178 raft_consensus.cc:2454] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fefe09513824e63b56e6a319338a3ab in term 1.
I20260810 00:55:31.375730 3272 leader_election.cc:302] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fefe09513824e63b56e6a319338a3ab, a65d29f653b84fa7a7dbb52e69b3f19f; no voters:
I20260810 00:55:31.375994 3469 raft_consensus.cc:2781] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:55:31.376451 3469 raft_consensus.cc:686] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [term 1 LEADER]: Becoming Leader. State: Replica: 5fefe09513824e63b56e6a319338a3ab, State: Running, Role: LEADER
I20260810 00:55:31.376889 3469 consensus_queue.cc:237] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:31.380735 2759 catalog_manager.cc:5495] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fefe09513824e63b56e6a319338a3ab (127.10.97.67). New cstate: current_term: 1 leader_uuid: "5fefe09513824e63b56e6a319338a3ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } health_report { overall_health: UNKNOWN } } }
I20260810 00:55:31.399963 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.394837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39398 (local address 127.10.97.67:37729)
0810 00:55:31.394997 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:31.395003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:31.395152 (+ 149us) server_negotiation.cc:407] Connection header received
0810 00:55:31.395370 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:31.395374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:31.395441 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:31.395558 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:55:31.395955 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.396650 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.397940 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.398329 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.398432 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:31.398974 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:55:31.399419 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:55:31.399540 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:31.399655 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 00:55:31.399728 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 00:55:31.407024 3178 raft_consensus.cc:1261] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11 [term 1 FOLLOWER]: Refusing update from remote peer 5fefe09513824e63b56e6a319338a3ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:55:31.407033 2963 raft_consensus.cc:1261] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Refusing update from remote peer 5fefe09513824e63b56e6a319338a3ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:55:31.407929 3477 consensus_queue.cc:1035] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:55:31.408228 3469 consensus_queue.cc:1035] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:55:31.424690 3485 mvcc.cc:205] Tried to move back new op lower bound from 7316780365435060224 to 7316780365323390976. Current Snapshot: MvccSnapshot[applied={T|T < 7316780365435060224}]
I20260810 00:55:31.430945 2755 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:55:31.431234 2755 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:55:31.434479 2755 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e: Sending DeleteTablet for 3 replicas of tablet 408b1aabf266427cb4e59b3cbc0fa92c
I20260810 00:55:31.435460 2933 tablet_service.cc:1514] Processing DeleteTablet for tablet 408b1aabf266427cb4e59b3cbc0fa92c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:55:31 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:50172
I20260810 00:55:31.435542 3340 tablet_service.cc:1514] Processing DeleteTablet for tablet 408b1aabf266427cb4e59b3cbc0fa92c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:55:31 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:39396
I20260810 00:55:31.435621 3150 tablet_service.cc:1514] Processing DeleteTablet for tablet 408b1aabf266427cb4e59b3cbc0fa92c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:55:31 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:42440
I20260810 00:55:31.436553 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 2814
I20260810 00:55:31.445726 3499 tablet_replica.cc:330] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab: stopping tablet replica
I20260810 00:55:31.446103 3499 raft_consensus.cc:2229] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:55:31.446729 3499 raft_consensus.cc:2258] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:55:31.448848 3499 ts_tablet_manager.cc:1905] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:55:31.454382 3450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.437398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50184 (local address 127.10.97.65:34371)
0810 00:55:31.437562 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:31.437568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:31.437588 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:55:31.437750 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:31.437756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:31.437859 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:31.437948 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:31.439748 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.440975 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.442932 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.443674 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.443819 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:31.447528 (+ 3709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:31.447541 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:31.447555 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:31.447596 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:31.451344 (+ 3748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:31.452179 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:31.452187 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:31.452190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:31.452354 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:31.452743 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:31.452748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:31.452751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:31.453249 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:31.453520 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:31.453826 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 00:55:31.454034 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 00:55:31.455080 3031 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:55:31.455538 3499 ts_tablet_manager.cc:1918] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:55:31.455713 3499 log.cc:1196] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/408b1aabf266427cb4e59b3cbc0fa92c
I20260810 00:55:31.456413 3499 ts_tablet_manager.cc:1939] T 408b1aabf266427cb4e59b3cbc0fa92c P 5fefe09513824e63b56e6a319338a3ab: Deleting consensus metadata
I20260810 00:55:31.458178 2736 catalog_manager.cc:4841] TS 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729): tablet 408b1aabf266427cb4e59b3cbc0fa92c (table test-table [id=6caa5ad8cae644988e17d12f84228908]) successfully deleted
I20260810 00:55:31.463229 3500 tablet_replica.cc:330] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11: stopping tablet replica
I20260810 00:55:31.463624 3500 raft_consensus.cc:2229] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:55:31.464145 3500 raft_consensus.cc:2258] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:55:31.466224 3500 ts_tablet_manager.cc:1905] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:55:31.472843 3500 ts_tablet_manager.cc:1918] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:55:31.472998 3500 log.cc:1196] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/408b1aabf266427cb4e59b3cbc0fa92c
I20260810 00:55:31.473536 3500 ts_tablet_manager.cc:1939] T 408b1aabf266427cb4e59b3cbc0fa92c P 3157f17735584b71a3eea3557e73df11: Deleting consensus metadata
I20260810 00:55:31.475169 2737 catalog_manager.cc:4841] TS 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591): tablet 408b1aabf266427cb4e59b3cbc0fa92c (table test-table [id=6caa5ad8cae644988e17d12f84228908]) successfully deleted
I20260810 00:55:31.594410 3498 tablet_replica.cc:330] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f: stopping tablet replica
I20260810 00:55:31.594841 3498 raft_consensus.cc:2229] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:55:31.595284 3498 raft_consensus.cc:2258] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:55:31.595644 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 2814
W20260810 00:55:31.628875 2735 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.97.65:34371 (error 108)
I20260810 00:55:31.630530 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 3045
I20260810 00:55:31.631443 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.630946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34371 (local address 127.0.0.1:50196)
0810 00:55:31.631151 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:31.631265 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260810 00:55:31.632015 2735 catalog_manager.cc:4584] TS a65d29f653b84fa7a7dbb52e69b3f19f (127.10.97.65:34371): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 408b1aabf266427cb4e59b3cbc0fa92c: Network error: Client connection negotiation failed: client connection to 127.10.97.65:34371: connect: Connection refused (error 111)
I20260810 00:55:31.645895 3455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.631773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42446 (local address 127.10.97.66:44591)
0810 00:55:31.631967 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:31.631973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:31.631993 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:55:31.632062 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:31.632067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:31.632151 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:31.632255 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:31.633837 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.635035 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.636460 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.637076 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.637224 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:31.639616 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:31.639634 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:31.639654 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:31.639707 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:31.643246 (+ 3539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:31.643908 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:31.643918 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:31.643922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:31.644084 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:31.644412 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:31.644417 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:31.644421 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:31.644880 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:31.645190 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:31.645345 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 00:55:31.645553 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 00:55:31.646575 3237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:55:31.685892 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.685350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34371 (local address 127.0.0.1:50208)
0810 00:55:31.685583 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:31.685702 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260810 00:55:31.687032 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.686635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34371 (local address 127.0.0.1:50212)
0810 00:55:31.686792 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:31.686894 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 00:55:31.733165 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 3045
I20260810 00:55:31.747678 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.747133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34371 (local address 127.0.0.1:36014)
0810 00:55:31.747388 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:31.747496 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260810 00:55:31.748922 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.748462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34371 (local address 127.0.0.1:36018)
0810 00:55:31.748686 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:31.748776 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260810 00:55:31.755272 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 3246
I20260810 00:55:31.770670 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.756141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51048 (local address 127.10.97.67:37729)
0810 00:55:31.756297 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:31.756301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:31.756322 (+ 21us) server_negotiation.cc:407] Connection header received
0810 00:55:31.756498 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:31.756502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:31.756564 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:31.756647 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:31.758639 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.759461 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.761270 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:31.761699 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:31.761791 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:31.764970 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:31.764981 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:31.764997 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:31.765047 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:31.767926 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:31.768733 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:31.768741 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:31.768743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:31.768889 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:31.769303 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:31.769307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:31.769309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:31.769660 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:31.769890 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:31.770230 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 00:55:31.770384 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 00:55:31.771440 3430 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:55:31.829262 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.828698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34371 (local address 127.0.0.1:36034)
0810 00:55:31.828939 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:31.829079 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 00:55:31.830349 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:31.829908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34371 (local address 127.0.0.1:36044)
0810 00:55:31.830125 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:31.830215 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260810 00:55:31.890198 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 3246
I20260810 00:55:31.914419 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 2711
I20260810 00:55:31.915493 2799 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:55:32.025288 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 2711
I20260810 00:55:32.025352 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:32.024448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:34371 (local address 127.0.0.1:36054)
0810 00:55:32.025012 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:32.025179 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402}
I20260810 00:55:52.052418 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:42075
--webserver_interface=127.10.97.126
--webserver_port=34067
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:39661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:42075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:55:52.371989 3511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:55:52.372236 3511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:55:52.372359 3511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:55:52.379297 3511 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 00:55:52.379384 3511 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 00:55:52.379451 3511 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 00:55:52.379516 3511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:55:52.379582 3511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:55:52.379681 3511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:55:52.379735 3511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:55:52.379823 3511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:55:52.389014 3511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:42075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:42075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=34067
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:55:52.390043 3511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:55:52.391256 3511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:55:52.401854 3520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:52.402004 3518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:52.401997 3524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:55:52.402724 3511 server_base.cc:1034] running on GCE node
I20260810 00:55:52.403426 3511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:55:52.404678 3511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:55:52.405893 3511 hybrid_clock.cc:648] HybridClock initialized: now 1786323352405837 us; error 26 us; skew 500 ppm
I20260810 00:55:52.406190 3511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:55:52 dist-test-slave-jlzn krb5kdc[1886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323352, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:55:52.411873 3511 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:55:52.413148 3511 webserver.cc:460] Webserver started at http://127.10.97.126:34067/ using document root <none> and password file <none>
I20260810 00:55:52.413769 3511 fs_manager.cc:359] Metadata directory not provided
I20260810 00:55:52.413869 3511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:55:52.418826 3511 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 00:55:52.421391 3534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:52.422521 3511 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:55:52.422672 3511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "8cc21197f71449078605f8078a16825e"
format_stamp: "Formatted at 2026-08-10 00:55:07 on dist-test-slave-jlzn"
server_key: "2221b4e1457a6a13623b621ffe498eb9"
server_key_iv: "197294f61c048fd25c902d27aa5528ea"
server_key_version: "encryptionkey@0"
I20260810 00:55:52.423544 3511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:55:52.440253 3511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:55:52.444079 3511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:55:52.444350 3511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:55:52.460393 3511 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:42075
I20260810 00:55:52.460448 3605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:42075 every 8 connection(s)
I20260810 00:55:52.461684 3511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:55:52.467846 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 3511
I20260810 00:55:52.468196 3606 sys_catalog.cc:263] Verifying existing consensus state
I20260810 00:55:52.468906 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:34371
--local_ip_for_outbound_sockets=127.10.97.65
--tserver_master_addrs=127.10.97.126:42075
--webserver_port=46255
--webserver_interface=127.10.97.65
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:39661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:55:52.472376 3606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e: Bootstrap starting.
I20260810 00:55:52.496284 3606 log.cc:824] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e: Log is configured to *not* fsync() on all Append() calls
I20260810 00:55:52.516629 3606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 00:55:52.517299 3606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e: Bootstrap complete.
I20260810 00:55:52.529934 3606 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc21197f71449078605f8078a16825e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42075 } }
I20260810 00:55:52.531114 3606 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cc21197f71449078605f8078a16825e, State: Initialized, Role: FOLLOWER
I20260810 00:55:52.531884 3606 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc21197f71449078605f8078a16825e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42075 } }
I20260810 00:55:52.532164 3606 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:55:52.532291 3606 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:55:52.532461 3606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 2 FOLLOWER]: Advancing to term 3
I20260810 00:55:52.537791 3606 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc21197f71449078605f8078a16825e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42075 } }
I20260810 00:55:52.538340 3606 leader_election.cc:302] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cc21197f71449078605f8078a16825e; no voters:
I20260810 00:55:52.538954 3606 leader_election.cc:288] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 00:55:52.539068 3611 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 3 FOLLOWER]: Leader election won for term 3
I20260810 00:55:52.540160 3611 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [term 3 LEADER]: Becoming Leader. State: Replica: 8cc21197f71449078605f8078a16825e, State: Running, Role: LEADER
I20260810 00:55:52.540694 3611 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc21197f71449078605f8078a16825e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42075 } }
I20260810 00:55:52.540957 3606 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:55:52.542989 3613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cc21197f71449078605f8078a16825e. Latest consensus state: current_term: 3 leader_uuid: "8cc21197f71449078605f8078a16825e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc21197f71449078605f8078a16825e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42075 } } }
I20260810 00:55:52.543318 3613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [sys.catalog]: This master's current role is: LEADER
I20260810 00:55:52.543835 3612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8cc21197f71449078605f8078a16825e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc21197f71449078605f8078a16825e" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42075 } } }
I20260810 00:55:52.544140 3612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e [sys.catalog]: This master's current role is: LEADER
I20260810 00:55:52.552340 3617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:55:52.560196 3617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:55:52.561605 3617 catalog_manager.cc:1259] Loaded cluster ID: b3a7e13690c849189952149cd0196486
I20260810 00:55:52.561751 3617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:55:52.565696 3617 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:55:52.567243 3617 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e: Loaded TSK: 0
I20260810 00:55:52.567442 3617 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8cc21197f71449078605f8078a16825e: Loaded TSK: 1
F20260810 00:55:52.568583 3617 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fabda5759ad google::LogMessage::Fail() at ??:0
@ 0x7fabda57888a google::LogMessage::SendToLog() at ??:0
@ 0x7fabda5754de google::LogMessage::Flush() at ??:0
@ 0x7fabda576a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fabde2d97d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fabde2c873e kudu::security::TokenSigner::ImportKeys() at ??:0
I20260810 00:55:52.589938 3637 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7fabeffa41c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fabeffa3783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fabf002f2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fabf002f0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fabf00795e9 std::function<>::operator()() at ??:0
@ 0x7fabeffa7cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fabeffa5c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fabf002d610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fabf002d439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fabdd1c1345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fabdd1c491c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fabdd1c4799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fabdd19feaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fabe04796db start_thread at ??:0
@ 0x7fabd940f61f clone at ??:0
W20260810 00:55:52.851066 3608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:55:52.851337 3608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:55:52.851452 3608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:55:52.858410 3608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:55:52.858516 3608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:55:52.858671 3608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:55:52.869660 3608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:34371
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=46255
--tserver_master_addrs=127.10.97.126:42075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:55:52.870770 3608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:55:52.872036 3608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:55:52.884156 3673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:52.884135 3675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:52.884132 3677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:55:52.885334 3608 server_base.cc:1034] running on GCE node
I20260810 00:55:52.885875 3608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:55:52.886760 3608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:55:52.887940 3608 hybrid_clock.cc:648] HybridClock initialized: now 1786323352887901 us; error 29 us; skew 500 ppm
I20260810 00:55:52.888213 3608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:55:52 dist-test-slave-jlzn krb5kdc[1886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323352, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:55:52.891054 3608 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:55:52.892416 3608 webserver.cc:460] Webserver started at http://127.10.97.65:46255/ using document root <none> and password file <none>
I20260810 00:55:52.892992 3608 fs_manager.cc:359] Metadata directory not provided
I20260810 00:55:52.893100 3608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:55:52.897951 3608 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 00:55:52.902005 3687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:52.903247 3608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 00:55:52.903384 3608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "a65d29f653b84fa7a7dbb52e69b3f19f"
format_stamp: "Formatted at 2026-08-10 00:55:07 on dist-test-slave-jlzn"
server_key: "b1b4613a97c41d82fe0b450fc5704b0a"
server_key_iv: "746e341fbb51fc81a6285609de39174e"
server_key_version: "encryptionkey@0"
I20260810 00:55:52.904345 3608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:55:52.925822 3608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:55:52.929271 3608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:55:52.929476 3608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:55:52.930317 3608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:55:52.931916 3695 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260810 00:55:52.937141 3608 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 00:55:52.937245 3608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260810 00:55:52.937361 3608 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 00:55:52.940282 3608 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 00:55:52.940358 3608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260810 00:55:52.940635 3695 tablet_bootstrap.cc:492] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f: Bootstrap starting.
I20260810 00:55:52.966120 3695 log.cc:824] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f: Log is configured to *not* fsync() on all Append() calls
I20260810 00:55:52.976362 3608 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:34371
I20260810 00:55:52.976398 3864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:34371 every 8 connection(s)
I20260810 00:55:52.977512 3608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:55:52.978281 3695 tablet_bootstrap.cc:492] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 00:55:52.979166 3695 tablet_bootstrap.cc:492] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f: Bootstrap complete.
I20260810 00:55:52.980473 3695 ts_tablet_manager.cc:1397] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f: Time spent bootstrapping tablet: real 0.040s user 0.021s sys 0.017s
I20260810 00:55:52.982378 3869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:52.980339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:41617)
0810 00:55:52.981200 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:52.981494 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":408,"threads_started":1}
W20260810 00:55:52.982947 3867 heartbeater.cc:644] Failed to heartbeat to 127.10.97.126:42075 (0 consecutive failures): Network error: Failed to ping master at 127.10.97.126:42075: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
I20260810 00:55:52.983781 3869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:52.983468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:42495)
0810 00:55:52.983605 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:52.983679 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 00:55:52.984687 3869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:52.984339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:33657)
0810 00:55:52.984473 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:52.984550 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 00:55:52.984920 3867 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 00:55:52.987854 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 3608
I20260810 00:55:52.987244 3695 raft_consensus.cc:348] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:52.988431 3695 raft_consensus.cc:729] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a65d29f653b84fa7a7dbb52e69b3f19f, State: Initialized, Role: FOLLOWER
I20260810 00:55:52.988664 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:44591
--local_ip_for_outbound_sockets=127.10.97.66
--tserver_master_addrs=127.10.97.126:42075
--webserver_port=44847
--webserver_interface=127.10.97.66
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:39661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:55:52.989037 3695 consensus_queue.cc:260] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:52.990368 3869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:52.990074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:41963)
0810 00:55:52.990204 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:52.990270 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 00:55:52.990527 3695 ts_tablet_manager.cc:1428] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f: Time spent starting tablet: real 0.010s user 0.009s sys 0.001s
I20260810 00:55:52.991317 3869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:52.991025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:54121)
0810 00:55:52.991151 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:52.991213 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 00:55:53.304397 3873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:55:53.304636 3873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:55:53.304764 3873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:55:53.311544 3873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:55:53.311637 3873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:55:53.311781 3873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:55:53.320693 3873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:44591
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=44847
--tserver_master_addrs=127.10.97.126:42075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:55:53.321735 3873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:55:53.322882 3873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:55:53.334957 3881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:53.334924 3885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:53.335362 3879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:55:53.335942 3873 server_base.cc:1034] running on GCE node
I20260810 00:55:53.336443 3873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:55:53.337213 3873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:55:53.338397 3873 hybrid_clock.cc:648] HybridClock initialized: now 1786323353338352 us; error 37 us; skew 500 ppm
I20260810 00:55:53.338649 3873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:55:53 dist-test-slave-jlzn krb5kdc[1886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323353, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:55:53.341502 3873 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:55:53.342756 3873 webserver.cc:460] Webserver started at http://127.10.97.66:44847/ using document root <none> and password file <none>
I20260810 00:55:53.343338 3873 fs_manager.cc:359] Metadata directory not provided
I20260810 00:55:53.343454 3873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:55:53.348331 3873 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 00:55:53.350723 3895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:53.351891 3873 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:55:53.352041 3873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "3157f17735584b71a3eea3557e73df11"
format_stamp: "Formatted at 2026-08-10 00:55:08 on dist-test-slave-jlzn"
server_key: "338825120f9c5a5043fba4bd0167b527"
server_key_iv: "325b92fd55f7e903ed4f64d6a4fd34b4"
server_key_version: "encryptionkey@0"
I20260810 00:55:53.352957 3873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:55:53.367426 3873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:55:53.370723 3873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:55:53.370956 3873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:55:53.371834 3873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:55:53.373049 3873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:55:53.373140 3873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:53.373240 3873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:55:53.373320 3873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:53.402307 3873 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:44591
I20260810 00:55:53.402367 4064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:44591 every 8 connection(s)
I20260810 00:55:53.403556 3873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:55:53.407895 4067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:53.406123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:48691)
0810 00:55:53.406777 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:53.407038 (+ 261us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":247,"threads_started":1}
W20260810 00:55:53.408522 4065 heartbeater.cc:644] Failed to heartbeat to 127.10.97.126:42075 (0 consecutive failures): Network error: Failed to ping master at 127.10.97.126:42075: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
I20260810 00:55:53.409210 4067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:53.408912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:55173)
0810 00:55:53.409031 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:53.409104 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 00:55:53.409935 4067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:53.409631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:32857)
0810 00:55:53.409764 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:53.409834 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 00:55:53.410141 4065 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 00:55:53.413736 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 3873
I20260810 00:55:53.414335 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:37729
--local_ip_for_outbound_sockets=127.10.97.67
--tserver_master_addrs=127.10.97.126:42075
--webserver_port=46395
--webserver_interface=127.10.97.67
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:39661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:55:53.726428 4070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:55:53.726677 4070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:55:53.726809 4070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:55:53.733928 4070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:55:53.734036 4070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:55:53.734192 4070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:55:53.743352 4070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:37729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=46395
--tserver_master_addrs=127.10.97.126:42075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:55:53.744516 4070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:55:53.745697 4070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:55:53.757777 4075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:53.757850 4076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:55:53.757826 4078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:55:53.760941 4070 server_base.cc:1034] running on GCE node
I20260810 00:55:53.761519 4070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:55:53.762349 4070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:55:53.763545 4070 hybrid_clock.cc:648] HybridClock initialized: now 1786323353763498 us; error 40 us; skew 500 ppm
I20260810 00:55:53.763850 4070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:55:53 dist-test-slave-jlzn krb5kdc[1886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323353, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:55:53.766681 4070 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:55:53.768100 4070 webserver.cc:460] Webserver started at http://127.10.97.67:46395/ using document root <none> and password file <none>
I20260810 00:55:53.768724 4070 fs_manager.cc:359] Metadata directory not provided
I20260810 00:55:53.768838 4070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:55:53.773610 4070 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 00:55:53.776022 4089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:53.777293 4070 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:55:53.777447 4070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "5fefe09513824e63b56e6a319338a3ab"
format_stamp: "Formatted at 2026-08-10 00:55:08 on dist-test-slave-jlzn"
server_key: "80d18bd3b04e11902af9621ab7186443"
server_key_iv: "47a67ddd023a7ac190e2783ae6bfe122"
server_key_version: "encryptionkey@0"
I20260810 00:55:53.778334 4070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:55:53.826335 4070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:55:53.829874 4070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:55:53.830109 4070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:55:53.830987 4070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:55:53.832247 4070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:55:53.832340 4070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:53.832441 4070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:55:53.832508 4070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:55:53.861166 4070 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:37729
I20260810 00:55:53.861238 4275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:37729 every 8 connection(s)
I20260810 00:55:53.862419 4070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:55:53.866760 4279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:53.865072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:49139)
0810 00:55:53.865757 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:53.866001 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":268,"threads_started":1}
W20260810 00:55:53.867355 4277 heartbeater.cc:644] Failed to heartbeat to 127.10.97.126:42075 (0 consecutive failures): Network error: Failed to ping master at 127.10.97.126:42075: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
I20260810 00:55:53.868093 4279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:53.867764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:38695)
0810 00:55:53.867929 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:53.867996 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 00:55:53.868826 4279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:53.868564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:49437)
0810 00:55:53.868663 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:53.868728 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 00:55:53.869036 4277 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 00:55:53.869130 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 4070
I20260810 00:55:53.992743 4281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:53.992049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:58703)
0810 00:55:53.992474 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:53.992562 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":212,"threads_started":1}
I20260810 00:55:54.411285 4282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:54.410555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:46983)
0810 00:55:54.410970 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:54.411061 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":198,"threads_started":1}
I20260810 00:55:54.499861 4283 raft_consensus.cc:482] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:55:54.500113 4283 raft_consensus.cc:504] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:54.501554 4283 leader_election.cc:288] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729), 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591)
Aug 10 00:55:54 dist-test-slave-jlzn krb5kdc[1886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323352, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.66@KRBTEST.COM
Aug 10 00:55:54 dist-test-slave-jlzn krb5kdc[1886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323352, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.67@KRBTEST.COM
I20260810 00:55:54.536286 4287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:54.502024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:56159 (local address 127.10.97.67:37729)
0810 00:55:54.517324 (+ 15300us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:54.517343 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:54.517580 (+ 237us) server_negotiation.cc:407] Connection header received
0810 00:55:54.519618 (+ 2038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:54.519627 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:54.520025 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:54.520276 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:54.522467 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:54.523625 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:54.525347 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:54.525957 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:54.526114 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:54.530220 (+ 4106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:54.530227 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:54.530238 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:54.530276 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:54.532947 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:54.533620 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:54.533626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:54.533631 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:54.533786 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:54.534147 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:54.534151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:54.534153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:54.534624 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:54.534805 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:54.535512 (+ 707us) server_negotiation.cc:299] Negotiation successful
0810 00:55:54.535882 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15088,"thread_start_us":15321,"threads_started":1}
I20260810 00:55:54.536290 4289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:54.502210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:52285 (local address 127.10.97.66:44591)
0810 00:55:54.518466 (+ 16256us) server_negotiation.cc:206] Beginning negotiation
0810 00:55:54.518480 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 00:55:54.518522 (+ 42us) server_negotiation.cc:407] Connection header received
0810 00:55:54.520154 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:55:54.520164 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:55:54.520559 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:55:54.520909 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:55:54.522466 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:54.523605 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:54.525345 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:55:54.525950 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:55:54.526110 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:54.529968 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:55:54.529975 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:55:54.529986 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:55:54.530019 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:55:54.532831 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:54.533535 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:54.533541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:54.533546 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:54.533712 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:55:54.534124 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:55:54.534129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:55:54.534131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:55:54.534665 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:55:54.534832 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 00:55:54.535517 (+ 685us) server_negotiation.cc:299] Negotiation successful
0810 00:55:54.535903 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16081,"thread_start_us":15618,"threads_started":1}
I20260810 00:55:54.536331 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:54.501803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:44591 (local address 127.10.97.65:52285)
0810 00:55:54.517117 (+ 15314us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:54.517160 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 00:55:54.518189 (+ 1029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:55:54.520863 (+ 2674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:55:54.520867 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:55:54.521547 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:55:54.522281 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:54.522302 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:54.523820 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:54.523826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:54.525212 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:54.525224 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:54.526175 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:54.526183 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:54.526459 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:54.527203 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:55:54.527221 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:55:54.529705 (+ 2484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:55:54.532960 (+ 3255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:55:54.532969 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:55:54.532982 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:55:54.533400 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:55:54.533820 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:55:54.533825 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:55:54.533829 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:55:54.534008 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:55:54.534783 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:55:54.534788 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:55:54.535044 (+ 256us) client_negotiation.cc:769] Sending connection context
0810 00:55:54.535262 (+ 218us) client_negotiation.cc:240] Negotiation successful
0810 00:55:54.535853 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15190,"mutex_wait_us":102,"spinlock_wait_cycles":10880,"thread_start_us":15817,"threads_started":1}
I20260810 00:55:54.536418 4286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:54.501846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:37729 (local address 127.10.97.65:56159)
0810 00:55:54.517381 (+ 15535us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:54.517396 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:55:54.517768 (+ 372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:55:54.520480 (+ 2712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:55:54.520492 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:55:54.521549 (+ 1057us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:55:54.522281 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:54.522301 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:54.523792 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:54.523796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:54.525214 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:55:54.525226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:55:54.526193 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:55:54.526201 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:55:54.526464 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:55:54.527145 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:55:54.527173 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:55:54.529995 (+ 2822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:55:54.533072 (+ 3077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:55:54.533078 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:55:54.533082 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:55:54.533496 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:55:54.533897 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:55:54.533902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:55:54.533905 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:55:54.534035 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:55:54.534754 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:55:54.534761 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:55:54.535046 (+ 285us) client_negotiation.cc:769] Sending connection context
0810 00:55:54.535262 (+ 216us) client_negotiation.cc:240] Negotiation successful
0810 00:55:54.536023 (+ 761us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15430,"mutex_wait_us":375,"spinlock_wait_cycles":69632,"thread_start_us":15082,"threads_started":1}
W20260810 00:55:54.539189 3691 leader_election.cc:341] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591): Not found: Tablet not found: 408b1aabf266427cb4e59b3cbc0fa92c
W20260810 00:55:54.539417 3690 leader_election.cc:341] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729): Not found: Tablet not found: 408b1aabf266427cb4e59b3cbc0fa92c
I20260810 00:55:54.539548 3690 leader_election.cc:302] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a65d29f653b84fa7a7dbb52e69b3f19f; no voters: 3157f17735584b71a3eea3557e73df11, 5fefe09513824e63b56e6a319338a3ab
I20260810 00:55:54.539949 4283 raft_consensus.cc:2726] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 00:55:54.870663 4292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:54.869538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:39033)
0810 00:55:54.870110 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:54.870339 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":260,"threads_started":1}
I20260810 00:55:54.993808 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:54.993447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:50941)
0810 00:55:54.993588 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:54.993677 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 00:55:55.412664 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:55.412044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:59759)
0810 00:55:55.412430 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:55.412504 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":206,"threads_started":1}
I20260810 00:55:55.872210 4296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:55.871559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:60789)
0810 00:55:55.871989 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:55.872065 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":239,"threads_started":1}
I20260810 00:55:55.995271 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:55.994527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:55293)
0810 00:55:55.995022 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:55.995104 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":223,"threads_started":1}
I20260810 00:55:56.199301 4299 raft_consensus.cc:482] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:55:56.199501 4299 raft_consensus.cc:504] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:56.200161 4299 leader_election.cc:288] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729), 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591)
W20260810 00:55:56.201119 3690 leader_election.cc:341] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729): Not found: Tablet not found: 408b1aabf266427cb4e59b3cbc0fa92c
W20260810 00:55:56.201296 3691 leader_election.cc:341] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591): Not found: Tablet not found: 408b1aabf266427cb4e59b3cbc0fa92c
I20260810 00:55:56.201408 3691 leader_election.cc:302] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a65d29f653b84fa7a7dbb52e69b3f19f; no voters: 3157f17735584b71a3eea3557e73df11, 5fefe09513824e63b56e6a319338a3ab
I20260810 00:55:56.201627 4299 raft_consensus.cc:2726] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 00:55:56.413985 4300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:56.413356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:33895)
0810 00:55:56.413753 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:56.413828 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":209,"threads_started":1}
I20260810 00:55:56.873549 4301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:56.872927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:45291)
0810 00:55:56.873329 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:56.873405 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":199,"threads_started":1}
I20260810 00:55:56.996649 4304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:56.996029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:50339)
0810 00:55:56.996419 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:56.996493 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":195,"threads_started":1}
I20260810 00:55:57.415359 4306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:57.414718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:58051)
0810 00:55:57.415115 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:57.415191 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":201,"threads_started":1}
I20260810 00:55:57.770036 4307 raft_consensus.cc:482] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:55:57.770226 4307 raft_consensus.cc:504] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:55:57.770885 4307 leader_election.cc:288] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729), 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591)
W20260810 00:55:57.771822 3691 leader_election.cc:341] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591): Not found: Tablet not found: 408b1aabf266427cb4e59b3cbc0fa92c
W20260810 00:55:57.772009 3690 leader_election.cc:341] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729): Not found: Tablet not found: 408b1aabf266427cb4e59b3cbc0fa92c
I20260810 00:55:57.772167 3690 leader_election.cc:302] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a65d29f653b84fa7a7dbb52e69b3f19f; no voters: 3157f17735584b71a3eea3557e73df11, 5fefe09513824e63b56e6a319338a3ab
I20260810 00:55:57.772398 4307 raft_consensus.cc:2726] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 00:55:57.874936 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:57.874260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:50125)
0810 00:55:57.874684 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:57.874759 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":211,"threads_started":1}
I20260810 00:55:57.998229 4310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:57.997516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:58759)
0810 00:55:57.997959 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:57.998045 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":206,"threads_started":1}
I20260810 00:55:58.416764 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:58.416107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:42641)
0810 00:55:58.416531 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:58.416607 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":216,"threads_started":1}
I20260810 00:55:58.876405 4313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:58.875733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:55209)
0810 00:55:58.876183 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:58.876259 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":255,"threads_started":1}
I20260810 00:55:58.999588 4314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:58.998960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:44359)
0810 00:55:58.999364 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:58.999441 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":213,"threads_started":1}
I20260810 00:55:59.418167 4315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:59.417491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:35093)
0810 00:55:59.417920 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:59.418000 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":231,"threads_started":1}
I20260810 00:55:59.877822 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:55:59.877171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:35369)
0810 00:55:59.877575 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0810 00:55:59.877663 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":205,"threads_started":1}
I20260810 00:56:00.001026 4319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:00.000366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:52685)
0810 00:56:00.000757 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:00.000846 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":202,"threads_started":1}
I20260810 00:56:00.419669 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:00.418917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:39395)
0810 00:56:00.419403 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:00.419488 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":225,"threads_started":1}
I20260810 00:56:00.879350 4321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:00.878563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:33357)
0810 00:56:00.879085 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:00.879170 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":249,"threads_started":1}
I20260810 00:56:01.002547 4322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:01.001789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:48079)
0810 00:56:01.002295 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:01.002400 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":271,"threads_started":1}
I20260810 00:56:01.421041 4324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:01.420417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:46267)
0810 00:56:01.420785 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:01.420876 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":191,"threads_started":1}
I20260810 00:56:01.466548 4325 raft_consensus.cc:482] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:56:01.466729 4325 raft_consensus.cc:504] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:56:01.467360 4325 leader_election.cc:288] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729), 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591)
W20260810 00:56:01.468233 3691 leader_election.cc:341] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591): Not found: Tablet not found: 408b1aabf266427cb4e59b3cbc0fa92c
W20260810 00:56:01.468401 3690 leader_election.cc:341] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729): Not found: Tablet not found: 408b1aabf266427cb4e59b3cbc0fa92c
I20260810 00:56:01.468514 3690 leader_election.cc:302] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a65d29f653b84fa7a7dbb52e69b3f19f; no voters: 3157f17735584b71a3eea3557e73df11, 5fefe09513824e63b56e6a319338a3ab
I20260810 00:56:01.468712 4325 raft_consensus.cc:2726] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 00:56:01.880808 4330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:01.880099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:50227)
0810 00:56:01.880513 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:01.880602 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":213,"threads_started":1}
I20260810 00:56:02.003979 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:02.003303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:48069)
0810 00:56:02.003702 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:02.003779 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":219,"threads_started":1}
I20260810 00:56:02.422470 4335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:02.421809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:33179)
0810 00:56:02.422242 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:02.422321 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":199,"threads_started":1}
I20260810 00:56:02.882298 4336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:02.881566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:45889)
0810 00:56:02.882032 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:02.882127 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":227,"threads_started":1}
I20260810 00:56:03.005421 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:03.004771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:51227)
0810 00:56:03.005181 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:03.005268 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":202,"threads_started":1}
I20260810 00:56:03.424010 4338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:03.423303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:38311)
0810 00:56:03.423729 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:03.423834 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":207,"threads_started":1}
I20260810 00:56:03.883752 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:03.882995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:43385)
0810 00:56:03.883428 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:03.883526 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":242,"threads_started":1}
I20260810 00:56:04.006875 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:04.006232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:37539)
0810 00:56:04.006613 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:04.006708 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":196,"threads_started":1}
I20260810 00:56:04.425458 4342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:04.424760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:49961)
0810 00:56:04.425210 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:04.425291 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":207,"threads_started":1}
I20260810 00:56:04.468281 4343 raft_consensus.cc:482] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:56:04.468487 4343 raft_consensus.cc:504] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:56:04.469230 4343 leader_election.cc:288] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729), 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591)
W20260810 00:56:04.470044 3690 leader_election.cc:341] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729): Not found: Tablet not found: 408b1aabf266427cb4e59b3cbc0fa92c
W20260810 00:56:04.470224 3691 leader_election.cc:341] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591): Not found: Tablet not found: 408b1aabf266427cb4e59b3cbc0fa92c
I20260810 00:56:04.470372 3691 leader_election.cc:302] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a65d29f653b84fa7a7dbb52e69b3f19f; no voters: 3157f17735584b71a3eea3557e73df11, 5fefe09513824e63b56e6a319338a3ab
I20260810 00:56:04.470619 4343 raft_consensus.cc:2726] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 00:56:04.885164 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:04.884528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:47371)
0810 00:56:04.884928 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:04.885017 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":198,"threads_started":1}
I20260810 00:56:05.008324 4349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:05.007579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:50205)
0810 00:56:05.008037 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:05.008145 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":261,"threads_started":1}
I20260810 00:56:05.426877 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:05.426208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:46429)
0810 00:56:05.426625 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:05.426714 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":206,"threads_started":1}
I20260810 00:56:05.886547 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:05.885913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:36327)
0810 00:56:05.886313 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:05.886387 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":217,"threads_started":1}
I20260810 00:56:06.009744 4355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:06.009085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:55201)
0810 00:56:06.009499 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:06.009579 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":208,"threads_started":1}
I20260810 00:56:06.195865 4357 raft_consensus.cc:482] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:56:06.196094 4357 raft_consensus.cc:504] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:56:06.196738 4357 leader_election.cc:288] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729), 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591)
W20260810 00:56:06.197733 3690 leader_election.cc:341] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729): Not found: Tablet not found: 408b1aabf266427cb4e59b3cbc0fa92c
W20260810 00:56:06.197908 3691 leader_election.cc:341] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591): Not found: Tablet not found: 408b1aabf266427cb4e59b3cbc0fa92c
I20260810 00:56:06.198056 3691 leader_election.cc:302] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a65d29f653b84fa7a7dbb52e69b3f19f; no voters: 3157f17735584b71a3eea3557e73df11, 5fefe09513824e63b56e6a319338a3ab
I20260810 00:56:06.198283 4357 raft_consensus.cc:2726] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 00:56:06.428298 4359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:06.427604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:33113)
0810 00:56:06.428068 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:06.428152 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":243,"threads_started":1}
I20260810 00:56:06.888005 4360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:06.887265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:58691)
0810 00:56:06.887726 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:06.887852 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":245,"threads_started":1}
I20260810 00:56:07.011158 4361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:07.010464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:54387)
0810 00:56:07.010868 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:07.010956 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":222,"threads_started":1}
I20260810 00:56:07.429757 4362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:07.429053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:43131)
0810 00:56:07.429498 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:07.429574 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":216,"threads_started":1}
I20260810 00:56:07.889386 4365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:07.888729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:48079)
0810 00:56:07.889122 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:07.889204 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":200,"threads_started":1}
I20260810 00:56:08.012754 4367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:08.012110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.65:48789)
0810 00:56:08.012516 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:08.012592 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":200,"threads_started":1}
I20260810 00:56:08.108255 4368 raft_consensus.cc:482] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:56:08.108435 4368 raft_consensus.cc:504] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65d29f653b84fa7a7dbb52e69b3f19f" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 34371 } } peers { permanent_uuid: "5fefe09513824e63b56e6a319338a3ab" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37729 } } peers { permanent_uuid: "3157f17735584b71a3eea3557e73df11" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 44591 } }
I20260810 00:56:08.109055 4368 leader_election.cc:288] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729), 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591)
W20260810 00:56:08.109937 3690 leader_election.cc:341] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fefe09513824e63b56e6a319338a3ab (127.10.97.67:37729): Not found: Tablet not found: 408b1aabf266427cb4e59b3cbc0fa92c
W20260810 00:56:08.110093 3691 leader_election.cc:341] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3157f17735584b71a3eea3557e73df11 (127.10.97.66:44591): Not found: Tablet not found: 408b1aabf266427cb4e59b3cbc0fa92c
I20260810 00:56:08.110217 3691 leader_election.cc:302] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a65d29f653b84fa7a7dbb52e69b3f19f; no voters: 3157f17735584b71a3eea3557e73df11, 5fefe09513824e63b56e6a319338a3ab
I20260810 00:56:08.110427 4368 raft_consensus.cc:2726] T 408b1aabf266427cb4e59b3cbc0fa92c P a65d29f653b84fa7a7dbb52e69b3f19f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 00:56:08.431358 4370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:08.430659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.66:56119)
0810 00:56:08.431075 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:08.431198 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":195,"threads_started":1}
I20260810 00:56:08.875630 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 3608
I20260810 00:56:08.889710 4371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:08.876667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41804 (local address 127.10.97.65:34371)
0810 00:56:08.877218 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:08.877229 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:08.877362 (+ 133us) server_negotiation.cc:407] Connection header received
0810 00:56:08.877568 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:08.877579 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:08.877869 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:08.878046 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:08.879434 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:08.880455 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:08.881663 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:08.882258 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:08.882353 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:08.884773 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:08.884784 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:08.884798 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:08.884834 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:08.887458 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:08.888077 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:08.888083 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:08.888086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:08.888236 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:08.888505 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:08.888508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:08.888510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:08.888828 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:08.889070 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:08.889261 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 00:56:08.889451 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":233,"threads_started":1}
I20260810 00:56:08.890985 3858 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:08.890996 4372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:08.890330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:42075 (local address 127.10.97.67:36561)
0810 00:56:08.890756 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:08.890854 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.126:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":215,"threads_started":1}
I20260810 00:56:08.975678 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 3608
I20260810 00:56:08.997215 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 3873
I20260810 00:56:09.009333 4374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:08.997889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59308 (local address 127.10.97.66:44591)
0810 00:56:08.998328 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:08.998332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:08.998349 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:56:08.998402 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:08.998405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:08.998465 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:08.998553 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:08.999882 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:09.000682 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:09.001929 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:09.002367 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:09.002459 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:09.004928 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:09.004933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:09.004936 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:09.004967 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:09.007260 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:09.007835 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:09.007840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:09.007844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:09.007961 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:09.008271 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:09.008274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:09.008276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:09.008513 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:09.008718 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:09.008888 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 00:56:09.009045 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":237,"threads_started":1}
I20260810 00:56:09.009979 4060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:09.089584 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 3873
I20260810 00:56:09.113343 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 4070
I20260810 00:56:09.126441 4379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:09.114109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43978 (local address 127.10.97.67:37729)
0810 00:56:09.114679 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:09.114684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:09.114700 (+ 16us) server_negotiation.cc:407] Connection header received
0810 00:56:09.114753 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:09.114757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:09.114821 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:09.114918 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:09.116363 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:09.117155 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:09.118356 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:09.118785 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:09.118893 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:09.121526 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:09.121531 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:09.121535 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:09.121566 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:09.123924 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:09.124572 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:09.124578 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:09.124582 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:09.124728 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:09.125084 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:09.125090 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:09.125093 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:09.125350 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:09.125610 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:09.125878 (+ 268us) server_negotiation.cc:299] Negotiation successful
0810 00:56:09.126084 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":276,"threads_started":1}
I20260810 00:56:09.127099 4269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:09.210770 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 4070
2026-08-10T00:56:09Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64013 ms)
[----------] 30 tests from SecurityITest (291807 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:56:09 dist-test-slave-jlzn krb5kdc[4383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:56:09 dist-test-slave-jlzn krb5kdc[4383](info): set up 2 sockets
Aug 10 00:56:09 dist-test-slave-jlzn krb5kdc[4383](info): commencing operation
krb5kdc: starting...
W20260810 00:56:10.421794 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.010s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:56:10 dist-test-slave-jlzn krb5kdc[4383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:56:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:56:10.658510 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:43369
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:35961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:43369
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:10.975835 4400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:10.976089 4400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:10.976218 4400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:10.984272 4400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:10.984386 4400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:10.984455 4400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:10.984539 4400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:10.984599 4400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:10.995491 4400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:35961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:43369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:43369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:10.996624 4400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:10.997845 4400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:11.008788 4414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:11.008821 4409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:11.008822 4411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:11.009315 4400 server_base.cc:1034] running on GCE node
I20260810 00:56:11.010099 4400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:11.011330 4400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:11.012523 4400 hybrid_clock.cc:648] HybridClock initialized: now 1786323371012473 us; error 40 us; skew 500 ppm
I20260810 00:56:11.012832 4400 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:11 dist-test-slave-jlzn krb5kdc[4383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323371, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:11.019904 4400 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:56:11.021206 4400 webserver.cc:460] Webserver started at http://127.10.97.126:43855/ using document root <none> and password file <none>
I20260810 00:56:11.021813 4400 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:11.021929 4400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:11.022214 4400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:11.024978 4400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "d236f9cf008348b593c6824ce12ca3cc"
format_stamp: "Formatted at 2026-08-10 00:56:11 on dist-test-slave-jlzn"
server_key: "ee0cc783c7807a5d69d144b68d209800"
server_key_iv: "a669a0275dd2c3d49af4e17629c34057"
server_key_version: "encryptionkey@0"
I20260810 00:56:11.025677 4400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "d236f9cf008348b593c6824ce12ca3cc"
format_stamp: "Formatted at 2026-08-10 00:56:11 on dist-test-slave-jlzn"
server_key: "ee0cc783c7807a5d69d144b68d209800"
server_key_iv: "a669a0275dd2c3d49af4e17629c34057"
server_key_version: "encryptionkey@0"
I20260810 00:56:11.030364 4400 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 00:56:11.033586 4426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:11.034792 4400 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260810 00:56:11.034942 4400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "d236f9cf008348b593c6824ce12ca3cc"
format_stamp: "Formatted at 2026-08-10 00:56:11 on dist-test-slave-jlzn"
server_key: "ee0cc783c7807a5d69d144b68d209800"
server_key_iv: "a669a0275dd2c3d49af4e17629c34057"
server_key_version: "encryptionkey@0"
I20260810 00:56:11.035112 4400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:11.050762 4400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:11.054342 4400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:11.054581 4400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:11.071076 4400 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:43369
I20260810 00:56:11.071110 4501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:43369 every 8 connection(s)
I20260810 00:56:11.072326 4400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:11.075791 4502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:11.078691 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 4400
I20260810 00:56:11.078907 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:11.079730 10629 external_mini_cluster.cc:1428] Setting key c426eda9edaa507743fb6e9ca70ab22a
I20260810 00:56:11.087849 4502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc: Bootstrap starting.
I20260810 00:56:11.091426 4502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:11.092701 4502 log.cc:824] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc: Log is configured to *not* fsync() on all Append() calls
Aug 10 00:56:11 dist-test-slave-jlzn krb5kdc[4383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:56:11.095501 4502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc: No bootstrap required, opened a new log
I20260810 00:56:11.103142 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:11.081654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52896 (local address 127.10.97.126:43369)
0810 00:56:11.082384 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:11.082395 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:11.082432 (+ 37us) server_negotiation.cc:407] Connection header received
0810 00:56:11.084087 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:11.084108 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:11.084468 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:11.084760 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:11.086523 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:11.087758 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:11.089719 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:11.090352 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:11.090476 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:11.095048 (+ 4572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:11.095060 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:11.095078 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:11.095120 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:11.098921 (+ 3801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:11.099540 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:11.099548 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:11.099556 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:11.099778 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:11.100147 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:11.100152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:11.100156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:11.100837 (+ 681us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:11.101139 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:11.101424 (+ 285us) server_negotiation.cc:299] Negotiation successful
0810 00:56:11.101860 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":329,"threads_started":1}
I20260810 00:56:11.105625 4502 raft_consensus.cc:348] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d236f9cf008348b593c6824ce12ca3cc" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43369 } }
I20260810 00:56:11.106037 4502 raft_consensus.cc:374] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:11.106127 4502 raft_consensus.cc:729] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d236f9cf008348b593c6824ce12ca3cc, State: Initialized, Role: FOLLOWER
I20260810 00:56:11.106613 4502 consensus_queue.cc:260] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d236f9cf008348b593c6824ce12ca3cc" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43369 } }
I20260810 00:56:11.106817 4502 raft_consensus.cc:388] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:11.106913 4502 raft_consensus.cc:482] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:11.107034 4502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:11.109066 4502 raft_consensus.cc:504] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d236f9cf008348b593c6824ce12ca3cc" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43369 } }
I20260810 00:56:11.109555 4502 leader_election.cc:302] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d236f9cf008348b593c6824ce12ca3cc; no voters:
I20260810 00:56:11.110174 4502 leader_election.cc:288] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:11.110327 4507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:11.111043 4507 raft_consensus.cc:686] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc [term 1 LEADER]: Becoming Leader. State: Replica: d236f9cf008348b593c6824ce12ca3cc, State: Running, Role: LEADER
I20260810 00:56:11.111536 4507 consensus_queue.cc:237] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d236f9cf008348b593c6824ce12ca3cc" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43369 } }
I20260810 00:56:11.112031 4502 sys_catalog.cc:564] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:11.114149 4508 sys_catalog.cc:455] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d236f9cf008348b593c6824ce12ca3cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d236f9cf008348b593c6824ce12ca3cc" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43369 } } }
I20260810 00:56:11.114552 4508 sys_catalog.cc:458] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:11.114732 4509 sys_catalog.cc:455] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader d236f9cf008348b593c6824ce12ca3cc. Latest consensus state: current_term: 1 leader_uuid: "d236f9cf008348b593c6824ce12ca3cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d236f9cf008348b593c6824ce12ca3cc" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 43369 } } }
I20260810 00:56:11.115216 4509 sys_catalog.cc:458] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:11.116567 4524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:11.121228 4524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:11.129765 4524 catalog_manager.cc:1347] Generated new cluster ID: d1f5d09d2c174fe7bc9aa4ce47376a4d
I20260810 00:56:11.129868 4524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:11.190687 4524 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:11.192047 4524 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:11.201059 4524 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc: Generated new TSK 0
I20260810 00:56:11.201987 4524 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
I20260810 00:56:11.284459 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:43369
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:35961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:11.653636 4541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:11.653898 4541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:11.654004 4541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:11.660887 4541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:11.660982 4541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:11.661105 4541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:56:11.670141 4541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:35961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:43369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:11.671219 4541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:11.672494 4541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:11.684599 4548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:11.684474 4549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:11.684593 4553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:11.685488 4541 server_base.cc:1034] running on GCE node
I20260810 00:56:11.686057 4541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:11.686856 4541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:11.688052 4541 hybrid_clock.cc:648] HybridClock initialized: now 1786323371688017 us; error 27 us; skew 500 ppm
I20260810 00:56:11.688313 4541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:11 dist-test-slave-jlzn krb5kdc[4383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323371, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:11.691336 4541 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:56:11.692680 4541 webserver.cc:460] Webserver started at http://127.10.97.65:39807/ using document root <none> and password file <none>
I20260810 00:56:11.693264 4541 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:11.693378 4541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:11.693684 4541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:11.696437 4541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "d55f5da40b8b4ece95b0d76b96c3dff6"
format_stamp: "Formatted at 2026-08-10 00:56:11 on dist-test-slave-jlzn"
server_key: "6d886a0aeda57f6d5c0727e2a9d3aa07"
server_key_iv: "4e36d75fb42a870220ad67abc7720535"
server_key_version: "encryptionkey@0"
I20260810 00:56:11.697167 4541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "d55f5da40b8b4ece95b0d76b96c3dff6"
format_stamp: "Formatted at 2026-08-10 00:56:11 on dist-test-slave-jlzn"
server_key: "6d886a0aeda57f6d5c0727e2a9d3aa07"
server_key_iv: "4e36d75fb42a870220ad67abc7720535"
server_key_version: "encryptionkey@0"
I20260810 00:56:11.701972 4541 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:56:11.705189 4564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:11.706624 4541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 00:56:11.706784 4541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "d55f5da40b8b4ece95b0d76b96c3dff6"
format_stamp: "Formatted at 2026-08-10 00:56:11 on dist-test-slave-jlzn"
server_key: "6d886a0aeda57f6d5c0727e2a9d3aa07"
server_key_iv: "4e36d75fb42a870220ad67abc7720535"
server_key_version: "encryptionkey@0"
I20260810 00:56:11.706982 4541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:11.719879 4541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:11.723278 4541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:11.723528 4541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:11.724403 4541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:56:11.725606 4541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:56:11.725677 4541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:11.725790 4541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:56:11.725843 4541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:11.755393 4541 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:40773
I20260810 00:56:11.755450 4722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:40773 every 8 connection(s)
I20260810 00:56:11.756589 4541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:56:11.759790 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 4541
I20260810 00:56:11.760012 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:56:11.760665 10629 external_mini_cluster.cc:1428] Setting key 47a24020c78f5547762d0dc883f9802d
Aug 10 00:56:11 dist-test-slave-jlzn krb5kdc[4383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323371, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:56:11.774864 4725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:11.758898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:40549 (local address 127.10.97.126:43369)
0810 00:56:11.759337 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:11.759341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:11.760204 (+ 863us) server_negotiation.cc:407] Connection header received
0810 00:56:11.762125 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:11.762130 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:11.762203 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:11.762312 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:11.763696 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:11.764451 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:11.765943 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:11.766342 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:11.766467 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:11.769676 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:11.769681 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:11.769684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:11.769714 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:11.772055 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:11.772705 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:11.772710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:11.772712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:11.772833 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:11.773239 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:11.773245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:11.773248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:11.773513 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:11.773648 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:11.774289 (+ 641us) server_negotiation.cc:299] Negotiation successful
0810 00:56:11.774522 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":234,"threads_started":1}
I20260810 00:56:11.775732 4727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:11.759266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43369 (local address 127.10.97.65:40549)
0810 00:56:11.760036 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:11.760092 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:11.761867 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:11.762520 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:11.762528 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:11.762988 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:56:11.763532 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:11.763550 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:11.764607 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:11.764611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:11.765803 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:11.765816 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:11.766560 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:11.766569 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:11.766792 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:11.767392 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:56:11.767413 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:56:11.769471 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:56:11.772180 (+ 2709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:56:11.772188 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:56:11.772202 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:56:11.772592 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:56:11.772935 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:56:11.772941 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:56:11.772944 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:56:11.773131 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:56:11.773636 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:56:11.773644 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:56:11.773928 (+ 284us) client_negotiation.cc:769] Sending connection context
0810 00:56:11.774161 (+ 233us) client_negotiation.cc:240] Negotiation successful
0810 00:56:11.774616 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":300,"threads_started":1}
I20260810 00:56:11.777115 4723 heartbeater.cc:347] Connected to a master server at 127.10.97.126:43369
I20260810 00:56:11.777411 4723 heartbeater.cc:464] Registering TS with master...
I20260810 00:56:11.778223 4723 heartbeater.cc:511] Master 127.10.97.126:43369 requested a full tablet report, sending...
I20260810 00:56:11.780493 4450 ts_manager.cc:194] Registered new tserver with Master: d55f5da40b8b4ece95b0d76b96c3dff6 (127.10.97.65:40773)
I20260810 00:56:11.782351 4450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:40549
WARNING: no policy specified for kudu/127.10.97.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
I20260810 00:56:11.820433 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:43369
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:35961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:12.128451 4731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:12.128705 4731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:12.128803 4731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:12.135644 4731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:12.135735 4731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:12.135880 4731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:56:12.144963 4731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:35961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:43369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:12.145980 4731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:12.147099 4731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:12.159062 4742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:12.159126 4741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:12.159101 4745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:12.160110 4731 server_base.cc:1034] running on GCE node
I20260810 00:56:12.160696 4731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:12.161510 4731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:12.162703 4731 hybrid_clock.cc:648] HybridClock initialized: now 1786323372162662 us; error 42 us; skew 500 ppm
I20260810 00:56:12.162977 4731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:12 dist-test-slave-jlzn krb5kdc[4383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323372, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:12.165890 4731 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:56:12.167202 4731 webserver.cc:460] Webserver started at http://127.10.97.66:33719/ using document root <none> and password file <none>
I20260810 00:56:12.167845 4731 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:12.167964 4731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:12.168258 4731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:12.170959 4731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "43bbc4b042f6432284fb96b4114e4cba"
format_stamp: "Formatted at 2026-08-10 00:56:12 on dist-test-slave-jlzn"
server_key: "dfd45b27dfb98dbd2f9c064bfc61887f"
server_key_iv: "d62d6aa2ef9270d1872b918465d87b5d"
server_key_version: "encryptionkey@0"
I20260810 00:56:12.171689 4731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "43bbc4b042f6432284fb96b4114e4cba"
format_stamp: "Formatted at 2026-08-10 00:56:12 on dist-test-slave-jlzn"
server_key: "dfd45b27dfb98dbd2f9c064bfc61887f"
server_key_iv: "d62d6aa2ef9270d1872b918465d87b5d"
server_key_version: "encryptionkey@0"
I20260810 00:56:12.176389 4731 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260810 00:56:12.179471 4753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:12.180680 4731 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:56:12.180823 4731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "43bbc4b042f6432284fb96b4114e4cba"
format_stamp: "Formatted at 2026-08-10 00:56:12 on dist-test-slave-jlzn"
server_key: "dfd45b27dfb98dbd2f9c064bfc61887f"
server_key_iv: "d62d6aa2ef9270d1872b918465d87b5d"
server_key_version: "encryptionkey@0"
I20260810 00:56:12.180989 4731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:12.196856 4731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:12.200268 4731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:12.200512 4731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:12.201437 4731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:56:12.202613 4731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:56:12.202698 4731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:12.202823 4731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:56:12.202877 4731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:12.232182 4731 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:41139
I20260810 00:56:12.232239 4916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:41139 every 8 connection(s)
I20260810 00:56:12.233317 4731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:56:12.241669 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 4731
I20260810 00:56:12.241833 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:56:12.242237 10629 external_mini_cluster.cc:1428] Setting key f5fe710df593a79705b62c61d64ba255
Aug 10 00:56:12 dist-test-slave-jlzn krb5kdc[4383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323372, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:56:12.251668 4725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:12.235536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:45561 (local address 127.10.97.126:43369)
0810 00:56:12.235693 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:12.235698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:12.236769 (+ 1071us) server_negotiation.cc:407] Connection header received
0810 00:56:12.238530 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:12.238535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:12.238616 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:12.238707 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:12.240208 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.240922 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.242465 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.242887 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.243002 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:12.246442 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:12.246446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:12.246450 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:12.246480 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:12.248894 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:12.249540 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:12.249545 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:12.249549 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:12.249687 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:12.250083 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:12.250087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:12.250089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:12.250339 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:12.250480 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:12.251074 (+ 594us) server_negotiation.cc:299] Negotiation successful
0810 00:56:12.251308 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 00:56:12.252483 4919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:12.235942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43369 (local address 127.10.97.66:45561)
0810 00:56:12.236607 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:12.236656 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:12.238289 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:12.238934 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:12.238942 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:12.239471 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:56:12.240047 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:12.240066 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.241065 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.241069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:12.242335 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:12.242351 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.243102 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.243110 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:12.243401 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:12.244088 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:56:12.244115 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:56:12.246242 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:56:12.249020 (+ 2778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:56:12.249029 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:56:12.249040 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:56:12.249420 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:56:12.249789 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:56:12.249793 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:56:12.249796 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:56:12.249969 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:56:12.250455 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:56:12.250462 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:56:12.250737 (+ 275us) client_negotiation.cc:769] Sending connection context
0810 00:56:12.250941 (+ 204us) client_negotiation.cc:240] Negotiation successful
0810 00:56:12.251396 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":258,"threads_started":1}
I20260810 00:56:12.253738 4917 heartbeater.cc:347] Connected to a master server at 127.10.97.126:43369
I20260810 00:56:12.253986 4917 heartbeater.cc:464] Registering TS with master...
I20260810 00:56:12.254690 4917 heartbeater.cc:511] Master 127.10.97.126:43369 requested a full tablet report, sending...
I20260810 00:56:12.256107 4450 ts_manager.cc:194] Registered new tserver with Master: 43bbc4b042f6432284fb96b4114e4cba (127.10.97.66:41139)
I20260810 00:56:12.257055 4450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:45561
WARNING: no policy specified for kudu/127.10.97.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
I20260810 00:56:12.300671 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:43369
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:35961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:12.610299 4923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:12.610563 4923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:12.610673 4923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:12.617614 4923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:12.617713 4923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:12.617823 4923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:56:12.626724 4923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:35961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:43369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:12.627731 4923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:12.628894 4923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:12.640574 4929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:12.640575 4939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:12.640636 4936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:12.641630 4923 server_base.cc:1034] running on GCE node
I20260810 00:56:12.642177 4923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:12.642983 4923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:12.644169 4923 hybrid_clock.cc:648] HybridClock initialized: now 1786323372644119 us; error 43 us; skew 500 ppm
I20260810 00:56:12.644439 4923 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:12 dist-test-slave-jlzn krb5kdc[4383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323372, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:12.647248 4923 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:56:12.648566 4923 webserver.cc:460] Webserver started at http://127.10.97.67:35979/ using document root <none> and password file <none>
I20260810 00:56:12.649154 4923 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:12.649271 4923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:12.649545 4923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:12.652279 4923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "2a82c9c00f624a7ab806228ec0733aae"
format_stamp: "Formatted at 2026-08-10 00:56:12 on dist-test-slave-jlzn"
server_key: "907cd294f25c4d7d995318ab45a00175"
server_key_iv: "6e1e97f2ec880ed986d6bea7eadf0d50"
server_key_version: "encryptionkey@0"
I20260810 00:56:12.652997 4923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "2a82c9c00f624a7ab806228ec0733aae"
format_stamp: "Formatted at 2026-08-10 00:56:12 on dist-test-slave-jlzn"
server_key: "907cd294f25c4d7d995318ab45a00175"
server_key_iv: "6e1e97f2ec880ed986d6bea7eadf0d50"
server_key_version: "encryptionkey@0"
I20260810 00:56:12.657727 4923 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:56:12.660812 4953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:12.661979 4923 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:56:12.662127 4923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "2a82c9c00f624a7ab806228ec0733aae"
format_stamp: "Formatted at 2026-08-10 00:56:12 on dist-test-slave-jlzn"
server_key: "907cd294f25c4d7d995318ab45a00175"
server_key_iv: "6e1e97f2ec880ed986d6bea7eadf0d50"
server_key_version: "encryptionkey@0"
I20260810 00:56:12.662290 4923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:12.681360 4923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:12.684955 4923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:12.685204 4923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:12.686023 4923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:56:12.687235 4923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:56:12.687310 4923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:12.687422 4923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:56:12.687472 4923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:12.714318 4923 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:44463
I20260810 00:56:12.714390 5122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:44463 every 8 connection(s)
I20260810 00:56:12.715468 4923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:56:12.721056 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 4923
I20260810 00:56:12.721215 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:56:12.721640 10629 external_mini_cluster.cc:1428] Setting key ba56f8bed8766757b37932816f8a2b5f
Aug 10 00:56:12 dist-test-slave-jlzn krb5kdc[4383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323372, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:56:12.733042 4725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:12.717686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:59681 (local address 127.10.97.126:43369)
0810 00:56:12.717846 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:12.717850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:12.718852 (+ 1002us) server_negotiation.cc:407] Connection header received
0810 00:56:12.720625 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:12.720631 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:12.720718 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:12.720802 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:12.722256 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.723126 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.725030 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.725453 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.725544 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:12.728432 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:12.728436 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:12.728439 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:12.728470 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:12.730673 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:12.731265 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:12.731269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:12.731271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:12.731389 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:12.731715 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:12.731718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:12.731720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:12.731994 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:12.732100 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:12.732657 (+ 557us) server_negotiation.cc:299] Negotiation successful
0810 00:56:12.732808 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 00:56:12.733829 5126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:12.718060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:43369 (local address 127.10.97.67:59681)
0810 00:56:12.718707 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:12.718741 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:12.720380 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:12.721028 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:12.721035 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:12.721538 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:56:12.722072 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:12.722095 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.723275 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.723281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:12.724897 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:12.724909 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.725613 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.725618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:12.725797 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:12.726295 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:56:12.726320 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:56:12.728260 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:56:12.730782 (+ 2522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:56:12.730788 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:56:12.730797 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:56:12.731180 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:56:12.731462 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:56:12.731465 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:56:12.731468 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:56:12.731636 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:56:12.732091 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:56:12.732096 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:56:12.732359 (+ 263us) client_negotiation.cc:769] Sending connection context
0810 00:56:12.732555 (+ 196us) client_negotiation.cc:240] Negotiation successful
0810 00:56:12.732870 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":251,"threads_started":1}
I20260810 00:56:12.735031 5124 heartbeater.cc:347] Connected to a master server at 127.10.97.126:43369
I20260810 00:56:12.735285 5124 heartbeater.cc:464] Registering TS with master...
I20260810 00:56:12.735947 5124 heartbeater.cc:511] Master 127.10.97.126:43369 requested a full tablet report, sending...
I20260810 00:56:12.737174 4450 ts_manager.cc:194] Registered new tserver with Master: 2a82c9c00f624a7ab806228ec0733aae (127.10.97.67:44463)
I20260810 00:56:12.737957 4450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:59681
I20260810 00:56:12.748291 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:56:12.765847 4725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:12.752539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56308 (local address 127.10.97.126:43369)
0810 00:56:12.752692 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:12.752696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:12.753148 (+ 452us) server_negotiation.cc:407] Connection header received
0810 00:56:12.753300 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:12.753303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:12.753356 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:12.753420 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:12.754872 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.755673 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.757581 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.758412 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.758539 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:12.761444 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:12.761448 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:12.761451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:12.761480 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:12.763750 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:12.764297 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:12.764300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:12.764302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:12.764424 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:12.764706 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:12.764709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:12.764711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:12.764959 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:12.765194 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:12.765437 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 00:56:12.765614 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 00:56:12.769338 4450 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:56:12.775758 4450 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:56:12.785444 4723 heartbeater.cc:503] Master 127.10.97.126:43369 was elected leader, sending a full tablet report...
I20260810 00:56:12.812999 5147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:12.787025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:40773 (local address 127.0.0.1:39940)
0810 00:56:12.798319 (+ 11294us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:12.798372 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:12.798524 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:12.806486 (+ 7962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:12.806491 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:12.806519 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:12.806883 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:12.806896 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.808735 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.808740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:12.810350 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:12.810363 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.812243 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.812249 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:12.812408 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:12.812441 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 00:56:12.812530 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 00:56:12.812646 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11073,"mutex_wait_us":24,"thread_start_us":10951,"threads_started":1}
I20260810 00:56:12.813093 5154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:12.799048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41920 (local address 127.10.97.67:44463)
0810 00:56:12.804332 (+ 5284us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:12.804341 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:12.804818 (+ 477us) server_negotiation.cc:407] Connection header received
0810 00:56:12.804921 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:12.804930 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:12.805161 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:12.805368 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:12.805952 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.807462 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.810239 (+ 2777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.811465 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.811592 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:12.811862 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:12.812623 (+ 761us) server_negotiation.cc:299] Negotiation successful
0810 00:56:12.812794 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5122,"thread_start_us":4965,"threads_started":1}
I20260810 00:56:12.813092 5150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:12.787679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39940 (local address 127.10.97.65:40773)
0810 00:56:12.804939 (+ 17260us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:12.804947 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:12.804968 (+ 21us) server_negotiation.cc:407] Connection header received
0810 00:56:12.805039 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:12.805044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:12.805223 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:12.807057 (+ 1834us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:12.807281 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.808569 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.810501 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.812017 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.812188 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:12.812451 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:12.812567 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 00:56:12.812784 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17101,"thread_start_us":16958,"threads_started":1}
I20260810 00:56:12.813452 5155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:12.800987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:44463 (local address 127.0.0.1:41920)
0810 00:56:12.804551 (+ 3564us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:12.804575 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:12.804700 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:12.805401 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:12.805409 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:12.805454 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:12.805779 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:12.805789 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.807603 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.807609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:12.810089 (+ 2480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:12.810102 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.811671 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.811678 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:12.811926 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:12.811976 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 00:56:12.812098 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 00:56:12.812282 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3449,"mutex_wait_us":44,"thread_start_us":3368,"threads_started":1}
I20260810 00:56:12.814857 5152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:12.787750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:41139 (local address 127.0.0.1:37924)
0810 00:56:12.800516 (+ 12766us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:12.800540 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:12.800650 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:12.806175 (+ 5525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:12.806177 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:12.806199 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:12.806671 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:12.806683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.808162 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.808167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:12.811980 (+ 3813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:12.811996 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.814155 (+ 2159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.814162 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:12.814317 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:12.814348 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 00:56:12.814437 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 00:56:12.814560 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12647,"mutex_wait_us":5,"thread_start_us":12973,"threads_started":1}
I20260810 00:56:12.814956 5153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:12.787792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37924 (local address 127.10.97.66:41139)
0810 00:56:12.805707 (+ 17915us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:12.805712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:12.805732 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:56:12.805809 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:12.805814 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:12.806005 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:12.806215 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:12.806845 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.808029 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.812173 (+ 4144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:12.813955 (+ 1782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:12.814087 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:12.814253 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:12.814491 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 00:56:12.814651 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17752,"thread_start_us":17641,"threads_started":1}
I20260810 00:56:12.817149 5022 tablet_service.cc:1467] Processing CreateTablet for tablet 777bde06048d4acb97002d17078038ac (DEFAULT_TABLE table=test-table [id=d2b636b9070145c88dbd00307588294b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:56:12.818356 5022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 777bde06048d4acb97002d17078038ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:12.818310 4823 tablet_service.cc:1467] Processing CreateTablet for tablet 777bde06048d4acb97002d17078038ac (DEFAULT_TABLE table=test-table [id=d2b636b9070145c88dbd00307588294b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:56:12.818310 4630 tablet_service.cc:1467] Processing CreateTablet for tablet 777bde06048d4acb97002d17078038ac (DEFAULT_TABLE table=test-table [id=d2b636b9070145c88dbd00307588294b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:56:12.819626 4823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 777bde06048d4acb97002d17078038ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:12.819633 4630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 777bde06048d4acb97002d17078038ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:12.828766 5156 tablet_bootstrap.cc:492] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae: Bootstrap starting.
I20260810 00:56:12.831712 5156 tablet_bootstrap.cc:654] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:12.832680 5156 log.cc:824] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:12.833487 5159 tablet_bootstrap.cc:492] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba: Bootstrap starting.
I20260810 00:56:12.834189 5158 tablet_bootstrap.cc:492] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6: Bootstrap starting.
I20260810 00:56:12.838081 5159 tablet_bootstrap.cc:654] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:12.838219 5158 tablet_bootstrap.cc:654] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:12.839202 5158 log.cc:824] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:12.839282 5159 log.cc:824] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:12.843319 5156 tablet_bootstrap.cc:492] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae: No bootstrap required, opened a new log
I20260810 00:56:12.843595 5156 ts_tablet_manager.cc:1397] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20260810 00:56:12.843755 5159 tablet_bootstrap.cc:492] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba: No bootstrap required, opened a new log
I20260810 00:56:12.844005 5159 ts_tablet_manager.cc:1397] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260810 00:56:12.844489 5158 tablet_bootstrap.cc:492] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6: No bootstrap required, opened a new log
I20260810 00:56:12.844738 5158 ts_tablet_manager.cc:1397] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260810 00:56:12.851049 5159 raft_consensus.cc:348] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55f5da40b8b4ece95b0d76b96c3dff6" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40773 } } peers { permanent_uuid: "43bbc4b042f6432284fb96b4114e4cba" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 41139 } } peers { permanent_uuid: "2a82c9c00f624a7ab806228ec0733aae" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 44463 } }
I20260810 00:56:12.851382 5159 raft_consensus.cc:374] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:12.851481 5159 raft_consensus.cc:729] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43bbc4b042f6432284fb96b4114e4cba, State: Initialized, Role: FOLLOWER
I20260810 00:56:12.852041 5159 consensus_queue.cc:260] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55f5da40b8b4ece95b0d76b96c3dff6" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40773 } } peers { permanent_uuid: "43bbc4b042f6432284fb96b4114e4cba" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 41139 } } peers { permanent_uuid: "2a82c9c00f624a7ab806228ec0733aae" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 44463 } }
I20260810 00:56:12.853017 4917 heartbeater.cc:503] Master 127.10.97.126:43369 was elected leader, sending a full tablet report...
I20260810 00:56:12.853648 5159 ts_tablet_manager.cc:1428] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260810 00:56:12.853787 5156 raft_consensus.cc:348] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55f5da40b8b4ece95b0d76b96c3dff6" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40773 } } peers { permanent_uuid: "43bbc4b042f6432284fb96b4114e4cba" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 41139 } } peers { permanent_uuid: "2a82c9c00f624a7ab806228ec0733aae" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 44463 } }
I20260810 00:56:12.854169 5156 raft_consensus.cc:374] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:12.854275 5156 raft_consensus.cc:729] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a82c9c00f624a7ab806228ec0733aae, State: Initialized, Role: FOLLOWER
I20260810 00:56:12.854725 5158 raft_consensus.cc:348] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55f5da40b8b4ece95b0d76b96c3dff6" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40773 } } peers { permanent_uuid: "43bbc4b042f6432284fb96b4114e4cba" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 41139 } } peers { permanent_uuid: "2a82c9c00f624a7ab806228ec0733aae" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 44463 } }
I20260810 00:56:12.854851 5156 consensus_queue.cc:260] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55f5da40b8b4ece95b0d76b96c3dff6" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40773 } } peers { permanent_uuid: "43bbc4b042f6432284fb96b4114e4cba" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 41139 } } peers { permanent_uuid: "2a82c9c00f624a7ab806228ec0733aae" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 44463 } }
I20260810 00:56:12.855109 5158 raft_consensus.cc:374] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:12.855229 5158 raft_consensus.cc:729] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55f5da40b8b4ece95b0d76b96c3dff6, State: Initialized, Role: FOLLOWER
I20260810 00:56:12.855794 5158 consensus_queue.cc:260] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55f5da40b8b4ece95b0d76b96c3dff6" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40773 } } peers { permanent_uuid: "43bbc4b042f6432284fb96b4114e4cba" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 41139 } } peers { permanent_uuid: "2a82c9c00f624a7ab806228ec0733aae" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 44463 } }
I20260810 00:56:12.856019 5124 heartbeater.cc:503] Master 127.10.97.126:43369 was elected leader, sending a full tablet report...
I20260810 00:56:12.856585 5156 ts_tablet_manager.cc:1428] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260810 00:56:12.858065 5158 ts_tablet_manager.cc:1428] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
W20260810 00:56:12.966925 5125 tablet.cc:2369] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:56:12.985016 4918 tablet.cc:2369] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:56:13.008669 4724 tablet.cc:2369] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:56:13.026434 5172 raft_consensus.cc:482] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:56:13.026684 5172 raft_consensus.cc:504] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55f5da40b8b4ece95b0d76b96c3dff6" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40773 } } peers { permanent_uuid: "43bbc4b042f6432284fb96b4114e4cba" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 41139 } } peers { permanent_uuid: "2a82c9c00f624a7ab806228ec0733aae" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 44463 } }
I20260810 00:56:13.028307 5172 leader_election.cc:288] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55f5da40b8b4ece95b0d76b96c3dff6 (127.10.97.65:40773), 43bbc4b042f6432284fb96b4114e4cba (127.10.97.66:41139)
I20260810 00:56:13.034653 5126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:13.028477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:40773 (local address 127.10.97.67:49153)
0810 00:56:13.028726 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:13.028745 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:13.028863 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:13.030009 (+ 1146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:13.030014 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:13.030038 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:13.030291 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:13.030301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:13.031353 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:13.031358 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:13.032801 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:13.032813 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:13.033985 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:13.033992 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:13.034187 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:13.034221 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 00:56:13.034303 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 00:56:13.034438 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":8}
I20260810 00:56:13.034711 5150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:13.028775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:49153 (local address 127.10.97.65:40773)
0810 00:56:13.029743 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:13.029750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:13.029769 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:56:13.029837 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:13.029842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:13.029904 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:13.029979 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:13.030434 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:13.031230 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:13.032959 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:13.033845 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:13.033959 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:13.034010 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:13.034351 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 00:56:13.034474 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 00:56:13.035115 5153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:13.028683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:51635 (local address 127.10.97.66:41139)
0810 00:56:13.028870 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:13.028876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:13.029642 (+ 766us) server_negotiation.cc:407] Connection header received
0810 00:56:13.029898 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:13.029902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:13.029964 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:13.030051 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:13.030714 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:13.031469 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:13.033340 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:13.034186 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:13.034339 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:13.034404 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:13.034750 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 00:56:13.034889 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 00:56:13.035158 5174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:13.028761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:41139 (local address 127.10.97.67:51635)
0810 00:56:13.029547 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:13.029565 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:13.029743 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:13.030096 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:13.030101 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:13.030146 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:13.030544 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:13.030557 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:13.031642 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:13.031646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:13.033186 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:13.033200 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:13.034386 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:13.034393 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:13.034557 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:13.034590 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 00:56:13.034692 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 00:56:13.034809 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":35,"spinlock_wait_cycles":2816,"thread_start_us":475,"threads_started":1}
I20260810 00:56:13.035765 4659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777bde06048d4acb97002d17078038ac" candidate_uuid: "2a82c9c00f624a7ab806228ec0733aae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55f5da40b8b4ece95b0d76b96c3dff6" is_pre_election: true
I20260810 00:56:13.035989 4852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777bde06048d4acb97002d17078038ac" candidate_uuid: "2a82c9c00f624a7ab806228ec0733aae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43bbc4b042f6432284fb96b4114e4cba" is_pre_election: true
I20260810 00:56:13.036314 4659 raft_consensus.cc:2454] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a82c9c00f624a7ab806228ec0733aae in term 0.
I20260810 00:56:13.036476 4852 raft_consensus.cc:2454] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a82c9c00f624a7ab806228ec0733aae in term 0.
I20260810 00:56:13.036940 4955 leader_election.cc:302] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a82c9c00f624a7ab806228ec0733aae, d55f5da40b8b4ece95b0d76b96c3dff6; no voters:
I20260810 00:56:13.037312 5172 raft_consensus.cc:2781] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:56:13.037441 5172 raft_consensus.cc:482] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:56:13.037544 5172 raft_consensus.cc:3037] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:13.039776 5172 raft_consensus.cc:504] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55f5da40b8b4ece95b0d76b96c3dff6" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40773 } } peers { permanent_uuid: "43bbc4b042f6432284fb96b4114e4cba" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 41139 } } peers { permanent_uuid: "2a82c9c00f624a7ab806228ec0733aae" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 44463 } }
I20260810 00:56:13.040330 5172 leader_election.cc:288] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [CANDIDATE]: Term 1 election: Requested vote from peers d55f5da40b8b4ece95b0d76b96c3dff6 (127.10.97.65:40773), 43bbc4b042f6432284fb96b4114e4cba (127.10.97.66:41139)
I20260810 00:56:13.040755 4852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777bde06048d4acb97002d17078038ac" candidate_uuid: "2a82c9c00f624a7ab806228ec0733aae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43bbc4b042f6432284fb96b4114e4cba"
I20260810 00:56:13.040860 4659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777bde06048d4acb97002d17078038ac" candidate_uuid: "2a82c9c00f624a7ab806228ec0733aae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55f5da40b8b4ece95b0d76b96c3dff6"
I20260810 00:56:13.040925 4852 raft_consensus.cc:3037] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:13.041065 4659 raft_consensus.cc:3037] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:13.042994 4852 raft_consensus.cc:2454] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a82c9c00f624a7ab806228ec0733aae in term 1.
I20260810 00:56:13.043255 4659 raft_consensus.cc:2454] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a82c9c00f624a7ab806228ec0733aae in term 1.
I20260810 00:56:13.043372 4956 leader_election.cc:302] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a82c9c00f624a7ab806228ec0733aae, 43bbc4b042f6432284fb96b4114e4cba; no voters:
I20260810 00:56:13.043673 5172 raft_consensus.cc:2781] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:13.044154 5172 raft_consensus.cc:686] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [term 1 LEADER]: Becoming Leader. State: Replica: 2a82c9c00f624a7ab806228ec0733aae, State: Running, Role: LEADER
I20260810 00:56:13.044584 5172 consensus_queue.cc:237] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55f5da40b8b4ece95b0d76b96c3dff6" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40773 } } peers { permanent_uuid: "43bbc4b042f6432284fb96b4114e4cba" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 41139 } } peers { permanent_uuid: "2a82c9c00f624a7ab806228ec0733aae" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 44463 } }
I20260810 00:56:13.048866 4450 catalog_manager.cc:5495] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a82c9c00f624a7ab806228ec0733aae (127.10.97.67). New cstate: current_term: 1 leader_uuid: "2a82c9c00f624a7ab806228ec0733aae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55f5da40b8b4ece95b0d76b96c3dff6" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43bbc4b042f6432284fb96b4114e4cba" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 41139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a82c9c00f624a7ab806228ec0733aae" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 44463 } health_report { overall_health: HEALTHY } } }
I20260810 00:56:13.085291 5154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:13.080288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41930 (local address 127.10.97.67:44463)
0810 00:56:13.080443 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:13.080448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:13.080464 (+ 16us) server_negotiation.cc:407] Connection header received
0810 00:56:13.080531 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:13.080535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:13.080599 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:13.080716 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:56:13.081089 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:13.081769 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:13.083167 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:13.083577 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:13.083708 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:13.084216 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:56:13.084756 (+ 540us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:56:13.084879 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:13.084987 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 00:56:13.085062 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 00:56:13.092633 4659 raft_consensus.cc:1261] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6 [term 1 FOLLOWER]: Refusing update from remote peer 2a82c9c00f624a7ab806228ec0733aae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:56:13.092624 4852 raft_consensus.cc:1261] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba [term 1 FOLLOWER]: Refusing update from remote peer 2a82c9c00f624a7ab806228ec0733aae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:56:13.093575 5175 consensus_queue.cc:1035] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55f5da40b8b4ece95b0d76b96c3dff6" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 40773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:56:13.093881 5172 consensus_queue.cc:1035] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [LEADER]: Connected to new peer: Peer: permanent_uuid: "43bbc4b042f6432284fb96b4114e4cba" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 41139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:56:13.105608 5180 mvcc.cc:205] Tried to move back new op lower bound from 7316780536178274304 to 7316780535993778176. Current Snapshot: MvccSnapshot[applied={T|T < 7316780536178274304}]
I20260810 00:56:13.116580 4449 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:56:13.116886 4449 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:56:13.120580 4449 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d236f9cf008348b593c6824ce12ca3cc: Sending DeleteTablet for 3 replicas of tablet 777bde06048d4acb97002d17078038ac
I20260810 00:56:13.121762 4630 tablet_service.cc:1514] Processing DeleteTablet for tablet 777bde06048d4acb97002d17078038ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:56:13 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:39940
I20260810 00:56:13.121764 5022 tablet_service.cc:1514] Processing DeleteTablet for tablet 777bde06048d4acb97002d17078038ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:56:13 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:41920
I20260810 00:56:13.121943 4823 tablet_service.cc:1514] Processing DeleteTablet for tablet 777bde06048d4acb97002d17078038ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:56:13 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:37924
I20260810 00:56:13.123107 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 4541
I20260810 00:56:13.141965 5195 tablet_replica.cc:330] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6: stopping tablet replica
I20260810 00:56:13.142371 5195 raft_consensus.cc:2229] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:56:13.142887 5195 raft_consensus.cc:2258] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:56:13.143668 5197 tablet_replica.cc:330] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae: stopping tablet replica
I20260810 00:56:13.144104 5197 raft_consensus.cc:2229] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:56:13.144778 5197 raft_consensus.cc:2258] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:56:13.145381 5198 tablet_replica.cc:330] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba: stopping tablet replica
Aug 10 00:56:13 dist-test-slave-jlzn krb5kdc[4383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:56:13.147037 5195 ts_tablet_manager.cc:1905] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:56:13.147557 5198 raft_consensus.cc:2229] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:56:13.148078 5198 raft_consensus.cc:2258] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:56:13.149058 5197 ts_tablet_manager.cc:1905] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:56:13.150113 5198 ts_tablet_manager.cc:1905] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:56:13.154708 5150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:13.124757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39942 (local address 127.10.97.65:40773)
0810 00:56:13.124931 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:13.124938 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:13.134910 (+ 9972us) server_negotiation.cc:407] Connection header received
0810 00:56:13.135314 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:13.135320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:13.135426 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:13.135497 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:13.137403 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:13.138542 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:13.140540 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:13.141270 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:13.141368 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:13.147306 (+ 5938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:13.147319 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:13.147334 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:13.147377 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:13.151011 (+ 3634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:13.151958 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:13.151967 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:13.151970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:13.152149 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:13.152636 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:13.152640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:13.152643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:13.153118 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:13.153418 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:13.154013 (+ 595us) server_negotiation.cc:299] Negotiation successful
0810 00:56:13.154305 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 00:56:13.155488 5195 ts_tablet_manager.cc:1918] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:56:13.155659 5195 log.cc:1196] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/777bde06048d4acb97002d17078038ac
I20260810 00:56:13.156286 5195 ts_tablet_manager.cc:1939] T 777bde06048d4acb97002d17078038ac P d55f5da40b8b4ece95b0d76b96c3dff6: Deleting consensus metadata
I20260810 00:56:13.156414 4716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:13.157223 5198 ts_tablet_manager.cc:1918] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:56:13.157435 5198 log.cc:1196] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/777bde06048d4acb97002d17078038ac
I20260810 00:56:13.157994 5198 ts_tablet_manager.cc:1939] T 777bde06048d4acb97002d17078038ac P 43bbc4b042f6432284fb96b4114e4cba: Deleting consensus metadata
I20260810 00:56:13.159732 4428 catalog_manager.cc:4841] TS 43bbc4b042f6432284fb96b4114e4cba (127.10.97.66:41139): tablet 777bde06048d4acb97002d17078038ac (table test-table [id=d2b636b9070145c88dbd00307588294b]) successfully deleted
I20260810 00:56:13.160244 5197 ts_tablet_manager.cc:1918] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:56:13.160446 5197 log.cc:1196] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/777bde06048d4acb97002d17078038ac
I20260810 00:56:13.160979 5197 ts_tablet_manager.cc:1939] T 777bde06048d4acb97002d17078038ac P 2a82c9c00f624a7ab806228ec0733aae: Deleting consensus metadata
I20260810 00:56:13.162595 4427 catalog_manager.cc:4841] TS 2a82c9c00f624a7ab806228ec0733aae (127.10.97.67:44463): tablet 777bde06048d4acb97002d17078038ac (table test-table [id=d2b636b9070145c88dbd00307588294b]) successfully deleted
I20260810 00:56:13.248972 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 4541
I20260810 00:56:13.249089 4427 catalog_manager.cc:4841] TS d55f5da40b8b4ece95b0d76b96c3dff6 (127.10.97.65:40773): tablet 777bde06048d4acb97002d17078038ac (table test-table [id=d2b636b9070145c88dbd00307588294b]) successfully deleted
I20260810 00:56:13.272198 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 4731
Aug 10 00:56:13 dist-test-slave-jlzn krb5kdc[4383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.66@KRBTEST.COM
I20260810 00:56:13.292083 5153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:13.272980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37936 (local address 127.10.97.66:41139)
0810 00:56:13.273179 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:13.273185 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:13.273206 (+ 21us) server_negotiation.cc:407] Connection header received
0810 00:56:13.273296 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:13.273302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:13.273388 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:13.273493 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:13.274705 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:13.275898 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:13.277255 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:13.277878 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:13.278001 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:13.286472 (+ 8471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:13.286484 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:13.286501 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:13.286539 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:13.289343 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:13.290115 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:13.290122 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:13.290132 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:13.290254 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:13.290646 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:13.290650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:13.290651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:13.291040 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:13.291265 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:13.291590 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 00:56:13.291753 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 00:56:13.292784 4912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:13.380837 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 4731
I20260810 00:56:13.401681 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 4923
Aug 10 00:56:13 dist-test-slave-jlzn krb5kdc[4383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.67@KRBTEST.COM
I20260810 00:56:13.418540 5154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:13.402652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41936 (local address 127.10.97.67:44463)
0810 00:56:13.402849 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:13.402855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:13.402877 (+ 22us) server_negotiation.cc:407] Connection header received
0810 00:56:13.403010 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:13.403015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:13.403104 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:13.403251 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:13.404646 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:13.405836 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:13.407080 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:13.407792 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:13.407954 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:13.412433 (+ 4479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:13.412460 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:13.412476 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:13.412529 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:13.415663 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:13.416514 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:13.416522 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:13.416524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:13.416685 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:13.417098 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:13.417102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:13.417104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:13.417465 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:13.417739 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:13.418033 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 00:56:13.418204 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 00:56:13.419260 5117 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:13.534780 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 4923
I20260810 00:56:13.555714 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 4400
I20260810 00:56:13.556814 4493 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:13.662724 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 4400
2026-08-10T00:56:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4455 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T00:56:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:13Z Disabled control of system clock
I20260810 00:56:13.749429 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:36253
--webserver_interface=127.10.97.126
--webserver_port=0
--builtin_ntp_servers=127.10.97.84:39967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:36253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 00:56:14.069206 5210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:14.069514 5210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:14.069607 5210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:14.076390 5210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:14.076478 5210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:14.076527 5210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:14.076582 5210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:14.076625 5210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:14.085228 5210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:36253
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:36253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:14.086236 5210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:14.087862 5210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:14.098441 5217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:14.098489 5220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:14.098536 5216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:14.099526 5210 server_base.cc:1034] running on GCE node
I20260810 00:56:14.100287 5210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:14.101496 5210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:14.102691 5210 hybrid_clock.cc:648] HybridClock initialized: now 1786323374102652 us; error 31 us; skew 500 ppm
I20260810 00:56:14.103001 5210 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:56:14.105069 5210 webserver.cc:460] Webserver started at http://127.10.97.126:46791/ using document root <none> and password file <none>
I20260810 00:56:14.105669 5210 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:14.105764 5210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:14.106067 5210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:14.108749 5210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "a8534e373a454874ad4d9457260399ef"
format_stamp: "Formatted at 2026-08-10 00:56:14 on dist-test-slave-jlzn"
server_key: "7452a2d525761edca1bc1e4d7d3251f9"
server_key_iv: "3db7346b357ae20a3502e35967cb8bba"
server_key_version: "encryptionkey@0"
I20260810 00:56:14.109496 5210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "a8534e373a454874ad4d9457260399ef"
format_stamp: "Formatted at 2026-08-10 00:56:14 on dist-test-slave-jlzn"
server_key: "7452a2d525761edca1bc1e4d7d3251f9"
server_key_iv: "3db7346b357ae20a3502e35967cb8bba"
server_key_version: "encryptionkey@0"
I20260810 00:56:14.114138 5210 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:56:14.117336 5229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:14.118489 5210 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:56:14.118662 5210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "a8534e373a454874ad4d9457260399ef"
format_stamp: "Formatted at 2026-08-10 00:56:14 on dist-test-slave-jlzn"
server_key: "7452a2d525761edca1bc1e4d7d3251f9"
server_key_iv: "3db7346b357ae20a3502e35967cb8bba"
server_key_version: "encryptionkey@0"
I20260810 00:56:14.118846 5210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:14.136689 5210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:14.137430 5210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:14.137683 5210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:14.153932 5210 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:36253
I20260810 00:56:14.153978 5299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:36253 every 8 connection(s)
I20260810 00:56:14.155085 5210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:14.158396 5300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:14.160801 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 5210
I20260810 00:56:14.160964 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:14.161476 10629 external_mini_cluster.cc:1428] Setting key 5e7888ff0f5c34f68b96346757187bd3
I20260810 00:56:14.171109 5300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef: Bootstrap starting.
I20260810 00:56:14.174693 5300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:14.175967 5300 log.cc:824] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:14.176424 5308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:14.163412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35368 (local address 127.10.97.126:36253)
0810 00:56:14.164468 (+ 1056us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:14.164481 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:14.164693 (+ 212us) server_negotiation.cc:407] Connection header received
0810 00:56:14.166852 (+ 2159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:14.166879 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:14.167437 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:14.167877 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:14.168395 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:14.170317 (+ 1922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:14.171634 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:14.172644 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:14.172838 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:14.173847 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:14.173858 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:14.173874 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:14.174514 (+ 640us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:14.174630 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:14.174657 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:14.174791 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:14.174925 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 00:56:14.175247 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":390,"threads_started":1}
I20260810 00:56:14.178682 5300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef: No bootstrap required, opened a new log
I20260810 00:56:14.185823 5300 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8534e373a454874ad4d9457260399ef" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 36253 } }
I20260810 00:56:14.186173 5300 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:14.186252 5300 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8534e373a454874ad4d9457260399ef, State: Initialized, Role: FOLLOWER
I20260810 00:56:14.186684 5300 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8534e373a454874ad4d9457260399ef" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 36253 } }
I20260810 00:56:14.186879 5300 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:14.186975 5300 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:14.187089 5300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:14.189038 5300 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8534e373a454874ad4d9457260399ef" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 36253 } }
I20260810 00:56:14.189451 5300 leader_election.cc:302] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8534e373a454874ad4d9457260399ef; no voters:
I20260810 00:56:14.190063 5300 leader_election.cc:288] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:14.190193 5310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:14.190838 5310 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef [term 1 LEADER]: Becoming Leader. State: Replica: a8534e373a454874ad4d9457260399ef, State: Running, Role: LEADER
I20260810 00:56:14.191314 5310 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8534e373a454874ad4d9457260399ef" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 36253 } }
I20260810 00:56:14.191897 5300 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:14.193933 5311 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8534e373a454874ad4d9457260399ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8534e373a454874ad4d9457260399ef" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 36253 } } }
I20260810 00:56:14.194347 5311 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:14.195209 5323 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:14.195087 5313 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8534e373a454874ad4d9457260399ef. Latest consensus state: current_term: 1 leader_uuid: "a8534e373a454874ad4d9457260399ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8534e373a454874ad4d9457260399ef" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 36253 } } }
I20260810 00:56:14.196503 5313 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:14.200337 5323 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:14.209321 5323 catalog_manager.cc:1347] Generated new cluster ID: 6dc2cae07a9c41d4a10873cef0093d9c
I20260810 00:56:14.209412 5323 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:14.216368 5323 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:14.217483 5323 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:14.233580 5323 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef: Generated new TSK 0
I20260810 00:56:14.234257 5323 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:56:14.242959 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:36253
--builtin_ntp_servers=127.10.97.84:39967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 00:56:14.584622 5341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:14.584865 5341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:14.584985 5341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:14.591750 5341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:14.591866 5341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:14.591977 5341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:56:14.601078 5341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:36253
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:14.602104 5341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:14.603693 5341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:14.615672 5349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:14.615720 5354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:14.615628 5347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:14.616552 5341 server_base.cc:1034] running on GCE node
I20260810 00:56:14.617105 5341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:14.617930 5341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:14.619124 5341 hybrid_clock.cc:648] HybridClock initialized: now 1786323374619080 us; error 38 us; skew 500 ppm
I20260810 00:56:14.619405 5341 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:56:14.621583 5341 webserver.cc:460] Webserver started at http://127.10.97.65:40785/ using document root <none> and password file <none>
I20260810 00:56:14.622189 5341 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:14.622291 5341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:14.622596 5341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:14.625253 5341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "2f7b8abb988b4425b0fcbd4226a3af46"
format_stamp: "Formatted at 2026-08-10 00:56:14 on dist-test-slave-jlzn"
server_key: "784016be9ee852701bb6e500441e84fb"
server_key_iv: "c0651ed664d718e36007fbaab57824c6"
server_key_version: "encryptionkey@0"
I20260810 00:56:14.625978 5341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "2f7b8abb988b4425b0fcbd4226a3af46"
format_stamp: "Formatted at 2026-08-10 00:56:14 on dist-test-slave-jlzn"
server_key: "784016be9ee852701bb6e500441e84fb"
server_key_iv: "c0651ed664d718e36007fbaab57824c6"
server_key_version: "encryptionkey@0"
I20260810 00:56:14.630541 5341 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 00:56:14.633694 5371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:14.634883 5341 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:56:14.635021 5341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "2f7b8abb988b4425b0fcbd4226a3af46"
format_stamp: "Formatted at 2026-08-10 00:56:14 on dist-test-slave-jlzn"
server_key: "784016be9ee852701bb6e500441e84fb"
server_key_iv: "c0651ed664d718e36007fbaab57824c6"
server_key_version: "encryptionkey@0"
I20260810 00:56:14.635212 5341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:14.660110 5341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:14.660784 5341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:14.661049 5341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:14.661891 5341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:56:14.663178 5341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:56:14.663271 5341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:14.663370 5341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:56:14.663443 5341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:14.699608 5341 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:36645
I20260810 00:56:14.699671 5536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:36645 every 8 connection(s)
I20260810 00:56:14.700798 5341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:56:14.706836 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 5341
I20260810 00:56:14.706982 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:56:14.707388 10629 external_mini_cluster.cc:1428] Setting key 526a3c94b4c2785a319ccf2a6e34aed1
I20260810 00:56:14.710042 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:36253
--builtin_ntp_servers=127.10.97.84:39967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 00:56:14.712459 5541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:14.703119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:53495 (local address 127.10.97.126:36253)
0810 00:56:14.703549 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:14.703555 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:14.704666 (+ 1111us) server_negotiation.cc:407] Connection header received
0810 00:56:14.706324 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:14.706328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:14.706390 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:14.706483 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:14.707497 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:14.708408 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:14.709878 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:14.710298 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:14.710407 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:14.711629 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:14.711640 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:14.711644 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:14.711661 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:14.711701 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:14.711712 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:14.711784 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:14.712027 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 00:56:14.712121 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":198,"threads_started":1}
I20260810 00:56:14.713286 5542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:14.703668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:36253 (local address 127.10.97.65:53495)
0810 00:56:14.704525 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:14.704559 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:14.706111 (+ 1552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:14.706674 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:14.706681 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:14.706765 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:56:14.707331 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:14.707349 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:14.708558 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:14.708561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:14.709741 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:14.709754 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:14.710509 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:14.710517 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:14.710739 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:14.711331 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:56:14.711334 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:56:14.711369 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:56:14.711439 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:56:14.711837 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:56:14.711848 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:56:14.711854 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 00:56:14.711999 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 00:56:14.712350 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":277,"threads_started":1}
I20260810 00:56:14.714483 5539 heartbeater.cc:347] Connected to a master server at 127.10.97.126:36253
I20260810 00:56:14.714746 5539 heartbeater.cc:464] Registering TS with master...
I20260810 00:56:14.715382 5539 heartbeater.cc:511] Master 127.10.97.126:36253 requested a full tablet report, sending...
I20260810 00:56:14.717218 5252 ts_manager.cc:194] Registered new tserver with Master: 2f7b8abb988b4425b0fcbd4226a3af46 (127.10.97.65:36645)
I20260810 00:56:14.718619 5252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.97.65:53495
W20260810 00:56:15.029022 5544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:15.029271 5544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:15.029356 5544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:15.036181 5544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:15.036263 5544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:15.036365 5544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:56:15.044799 5544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:36253
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:15.045866 5544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:15.047464 5544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:15.060335 5553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:15.060366 5552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:15.060297 5555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:15.061010 5544 server_base.cc:1034] running on GCE node
I20260810 00:56:15.061556 5544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:15.062372 5544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:15.063568 5544 hybrid_clock.cc:648] HybridClock initialized: now 1786323375063521 us; error 39 us; skew 500 ppm
I20260810 00:56:15.063885 5544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:56:15.065896 5544 webserver.cc:460] Webserver started at http://127.10.97.66:44849/ using document root <none> and password file <none>
I20260810 00:56:15.066501 5544 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:15.066593 5544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:15.066869 5544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:15.069584 5544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "1c2156e9807e4e9facd27fb69f3b0ba4"
format_stamp: "Formatted at 2026-08-10 00:56:15 on dist-test-slave-jlzn"
server_key: "e1f38da6736561d6d2761eb49b4ca7cc"
server_key_iv: "fb2c50539daf83dab4d63c725cdac6a0"
server_key_version: "encryptionkey@0"
I20260810 00:56:15.070320 5544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "1c2156e9807e4e9facd27fb69f3b0ba4"
format_stamp: "Formatted at 2026-08-10 00:56:15 on dist-test-slave-jlzn"
server_key: "e1f38da6736561d6d2761eb49b4ca7cc"
server_key_iv: "fb2c50539daf83dab4d63c725cdac6a0"
server_key_version: "encryptionkey@0"
I20260810 00:56:15.074898 5544 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260810 00:56:15.077960 5562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:15.079170 5544 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:56:15.079312 5544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "1c2156e9807e4e9facd27fb69f3b0ba4"
format_stamp: "Formatted at 2026-08-10 00:56:15 on dist-test-slave-jlzn"
server_key: "e1f38da6736561d6d2761eb49b4ca7cc"
server_key_iv: "fb2c50539daf83dab4d63c725cdac6a0"
server_key_version: "encryptionkey@0"
I20260810 00:56:15.079495 5544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:15.095553 5544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:15.096279 5544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:15.096537 5544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:15.097429 5544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:56:15.098673 5544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:56:15.098769 5544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:15.098868 5544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:56:15.098939 5544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:15.139575 5544 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:34257
I20260810 00:56:15.139662 5729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:34257 every 8 connection(s)
I20260810 00:56:15.141110 5544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:56:15.147038 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 5544
I20260810 00:56:15.147212 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:56:15.147660 10629 external_mini_cluster.cc:1428] Setting key cbd9a78c594f4bfcf85c349eb1668de6
I20260810 00:56:15.150805 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:36253
--builtin_ntp_servers=127.10.97.84:39967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 00:56:15.161423 5541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:15.143988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:57057 (local address 127.10.97.126:36253)
0810 00:56:15.144200 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:15.144206 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:15.145731 (+ 1525us) server_negotiation.cc:407] Connection header received
0810 00:56:15.148306 (+ 2575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:15.148312 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:15.148400 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:15.148498 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:15.149605 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.150611 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.152964 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.153503 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.153635 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:15.160616 (+ 6981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:15.160622 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:15.160628 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:15.160649 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:15.160689 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:15.160704 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:15.160795 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:15.161009 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 00:56:15.161102 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 00:56:15.162621 5732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:15.144493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:36253 (local address 127.10.97.66:57057)
0810 00:56:15.145559 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:15.145607 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:15.147956 (+ 2349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:15.148745 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:15.148753 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:15.148835 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:56:15.149399 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:15.149424 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.150795 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.150800 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:15.152826 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:15.152843 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.153878 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.153884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:15.154142 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:15.160273 (+ 6131us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:56:15.160276 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:56:15.160324 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:56:15.160416 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:56:15.160807 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:56:15.160817 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:56:15.160824 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 00:56:15.160983 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 00:56:15.161321 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":305,"threads_started":1}
I20260810 00:56:15.163949 5730 heartbeater.cc:347] Connected to a master server at 127.10.97.126:36253
I20260810 00:56:15.164222 5730 heartbeater.cc:464] Registering TS with master...
I20260810 00:56:15.164901 5730 heartbeater.cc:511] Master 127.10.97.126:36253 requested a full tablet report, sending...
I20260810 00:56:15.166270 5252 ts_manager.cc:194] Registered new tserver with Master: 1c2156e9807e4e9facd27fb69f3b0ba4 (127.10.97.66:34257)
I20260810 00:56:15.167330 5252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.97.66:57057
W20260810 00:56:15.472231 5733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:15.472470 5733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:15.472558 5733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:15.479302 5733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:15.479386 5733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:15.479488 5733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:56:15.488077 5733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:36253
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:15.489070 5733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:15.490660 5733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:15.502727 5743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:15.502763 5740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:15.502799 5739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:15.503340 5733 server_base.cc:1034] running on GCE node
I20260810 00:56:15.504021 5733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:15.504840 5733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:15.506038 5733 hybrid_clock.cc:648] HybridClock initialized: now 1786323375505989 us; error 43 us; skew 500 ppm
I20260810 00:56:15.506325 5733 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:56:15.508358 5733 webserver.cc:460] Webserver started at http://127.10.97.67:36003/ using document root <none> and password file <none>
I20260810 00:56:15.508965 5733 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:15.509064 5733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:15.509361 5733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:15.512030 5733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "909d3151e0424fc49d26b96d6f800c55"
format_stamp: "Formatted at 2026-08-10 00:56:15 on dist-test-slave-jlzn"
server_key: "282fbf533655a1dfe4fbf718b65eb30a"
server_key_iv: "0d0eaf9d1b9e51540800ddf7e8bdaad4"
server_key_version: "encryptionkey@0"
I20260810 00:56:15.512775 5733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "909d3151e0424fc49d26b96d6f800c55"
format_stamp: "Formatted at 2026-08-10 00:56:15 on dist-test-slave-jlzn"
server_key: "282fbf533655a1dfe4fbf718b65eb30a"
server_key_iv: "0d0eaf9d1b9e51540800ddf7e8bdaad4"
server_key_version: "encryptionkey@0"
I20260810 00:56:15.517408 5733 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:56:15.520534 5751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:15.521689 5733 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:56:15.521836 5733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "909d3151e0424fc49d26b96d6f800c55"
format_stamp: "Formatted at 2026-08-10 00:56:15 on dist-test-slave-jlzn"
server_key: "282fbf533655a1dfe4fbf718b65eb30a"
server_key_iv: "0d0eaf9d1b9e51540800ddf7e8bdaad4"
server_key_version: "encryptionkey@0"
I20260810 00:56:15.522020 5733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:15.540802 5733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:15.541503 5733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:15.541770 5733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:15.542629 5733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:56:15.543941 5733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:56:15.544035 5733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:15.544144 5733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:56:15.544220 5733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:15.572865 5733 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:36379
I20260810 00:56:15.572932 5915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:36379 every 8 connection(s)
I20260810 00:56:15.573969 5733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:56:15.581740 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 5733
I20260810 00:56:15.581898 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:56:15.582338 10629 external_mini_cluster.cc:1428] Setting key 020595791c7f8bf5ced1dd329c749920
I20260810 00:56:15.585566 5541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:15.576252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:43891 (local address 127.10.97.126:36253)
0810 00:56:15.576429 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:15.576433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:15.577787 (+ 1354us) server_negotiation.cc:407] Connection header received
0810 00:56:15.579390 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:15.579395 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:15.579463 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:15.579549 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:15.580477 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.581200 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.582836 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.583432 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.583543 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:15.584891 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:15.584898 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:15.584903 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:15.584924 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:15.584960 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:15.584973 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:15.585048 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:15.585242 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 00:56:15.585313 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 00:56:15.586378 5921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:15.576729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:36253 (local address 127.10.97.67:43891)
0810 00:56:15.577634 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:15.577678 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:15.579172 (+ 1494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:15.579708 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:15.579714 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:15.579788 (+ 74us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:56:15.580352 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:15.580369 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.581313 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.581317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:15.582678 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:15.582695 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.583624 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.583630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:15.583891 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:15.584542 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:56:15.584545 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:56:15.584593 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:56:15.584688 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:56:15.585063 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:56:15.585074 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:56:15.585080 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 00:56:15.585219 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 00:56:15.585467 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":257,"threads_started":1}
I20260810 00:56:15.587498 5916 heartbeater.cc:347] Connected to a master server at 127.10.97.126:36253
I20260810 00:56:15.587740 5916 heartbeater.cc:464] Registering TS with master...
I20260810 00:56:15.588406 5916 heartbeater.cc:511] Master 127.10.97.126:36253 requested a full tablet report, sending...
I20260810 00:56:15.589587 5252 ts_manager.cc:194] Registered new tserver with Master: 909d3151e0424fc49d26b96d6f800c55 (127.10.97.67:36379)
I20260810 00:56:15.590327 5252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.97.67:43891
I20260810 00:56:15.596909 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:56:15.605664 5541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:15.600722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35376 (local address 127.10.97.126:36253)
0810 00:56:15.600850 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:15.600854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:15.601299 (+ 445us) server_negotiation.cc:407] Connection header received
0810 00:56:15.601415 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:15.601419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:15.601479 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:15.601553 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:15.601938 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.602557 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.603789 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.604191 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.604275 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:15.605196 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:15.605200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:15.605204 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:15.605221 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:15.605248 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:15.605259 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:15.605322 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:15.605399 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 00:56:15.605464 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 00:56:15.608868 5252 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:56:15.615257 5252 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:56:15.647907 5936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:15.626183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58450 (local address 127.10.97.67:36379)
0810 00:56:15.637413 (+ 11230us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:15.637422 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:15.641185 (+ 3763us) server_negotiation.cc:407] Connection header received
0810 00:56:15.641522 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:15.641529 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:15.641748 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:15.641929 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:15.642890 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.644177 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.645926 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.646778 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.646895 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:15.647179 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:15.647383 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 00:56:15.647540 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11070,"thread_start_us":10918,"threads_started":1}
I20260810 00:56:15.647907 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:15.625945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:36379 (local address 127.0.0.1:58450)
0810 00:56:15.641040 (+ 15095us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:15.641085 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:15.641335 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:15.642197 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:15.642202 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:15.642229 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:15.642714 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:15.642728 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.644298 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.644304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:15.645766 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:15.645779 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.646971 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.646978 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:15.647162 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:15.647216 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 00:56:15.647329 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 00:56:15.647511 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14825,"mutex_wait_us":35,"spinlock_wait_cycles":11520,"thread_start_us":14755,"threads_started":1}
I20260810 00:56:15.651531 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:15.627059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60062 (local address 127.10.97.66:34257)
0810 00:56:15.640373 (+ 13314us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:15.640393 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:15.641279 (+ 886us) server_negotiation.cc:407] Connection header received
0810 00:56:15.641521 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:15.641529 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:15.641748 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:15.641933 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:15.643237 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.644826 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.648949 (+ 4123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.650280 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.650382 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:15.650629 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:15.651155 (+ 526us) server_negotiation.cc:299] Negotiation successful
0810 00:56:15.651267 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13129,"thread_start_us":13614,"threads_started":1}
I20260810 00:56:15.651558 5935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:15.626662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:34257 (local address 127.0.0.1:60062)
0810 00:56:15.641182 (+ 14520us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:15.641198 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:15.641335 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:15.641993 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:15.642000 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:15.642029 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:15.643028 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:15.643039 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.645029 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.645035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:15.647314 (+ 2279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:15.647329 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.650797 (+ 3468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.650803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:15.650975 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:15.651005 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 00:56:15.651096 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 00:56:15.651223 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14378,"mutex_wait_us":3,"thread_start_us":13880,"threads_started":1}
I20260810 00:56:15.651829 5827 tablet_service.cc:1467] Processing CreateTablet for tablet 88238dfff9ef41fca85c5d1bc14f2de4 (DEFAULT_TABLE table=test-table [id=9ff6715b58274cb38eeae5afaef06736]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:56:15.653236 5827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88238dfff9ef41fca85c5d1bc14f2de4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:15.654618 5633 tablet_service.cc:1467] Processing CreateTablet for tablet 88238dfff9ef41fca85c5d1bc14f2de4 (DEFAULT_TABLE table=test-table [id=9ff6715b58274cb38eeae5afaef06736]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:56:15.655884 5633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88238dfff9ef41fca85c5d1bc14f2de4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:15.657910 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:15.641713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:36645 (local address 127.0.0.1:47538)
0810 00:56:15.642575 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:15.642592 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:15.642708 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:15.648673 (+ 5965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:15.648678 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:15.648701 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:15.651938 (+ 3237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:15.651950 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.653541 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.653546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:15.655382 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:15.655398 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.657195 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.657202 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:15.657369 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:15.657399 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 00:56:15.657502 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 00:56:15.657622 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":1150,"threads_started":1}
I20260810 00:56:15.658033 5943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:15.642012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47538 (local address 127.10.97.65:36645)
0810 00:56:15.647876 (+ 5864us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:15.647882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:15.647905 (+ 23us) server_negotiation.cc:407] Connection header received
0810 00:56:15.647980 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:15.647987 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:15.648493 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:15.648661 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:15.652087 (+ 3426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.653387 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.655609 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.656991 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.657143 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:15.657354 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:15.657539 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 00:56:15.657701 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5697,"thread_start_us":5589,"threads_started":1}
I20260810 00:56:15.661464 5439 tablet_service.cc:1467] Processing CreateTablet for tablet 88238dfff9ef41fca85c5d1bc14f2de4 (DEFAULT_TABLE table=test-table [id=9ff6715b58274cb38eeae5afaef06736]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:56:15.662706 5439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88238dfff9ef41fca85c5d1bc14f2de4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:15.665671 5944 tablet_bootstrap.cc:492] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55: Bootstrap starting.
I20260810 00:56:15.668658 5944 tablet_bootstrap.cc:654] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:15.669020 5945 tablet_bootstrap.cc:492] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4: Bootstrap starting.
I20260810 00:56:15.669996 5944 log.cc:824] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:15.671969 5945 tablet_bootstrap.cc:654] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:15.673106 5944 tablet_bootstrap.cc:492] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55: No bootstrap required, opened a new log
I20260810 00:56:15.673120 5945 log.cc:824] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:15.673388 5944 ts_tablet_manager.cc:1397] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260810 00:56:15.674662 5950 tablet_bootstrap.cc:492] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46: Bootstrap starting.
I20260810 00:56:15.677872 5950 tablet_bootstrap.cc:654] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:15.679024 5950 log.cc:824] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:15.682024 5950 tablet_bootstrap.cc:492] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46: No bootstrap required, opened a new log
I20260810 00:56:15.682036 5945 tablet_bootstrap.cc:492] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4: No bootstrap required, opened a new log
I20260810 00:56:15.682283 5950 ts_tablet_manager.cc:1397] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260810 00:56:15.682283 5945 ts_tablet_manager.cc:1397] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260810 00:56:15.683346 5944 raft_consensus.cc:348] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909d3151e0424fc49d26b96d6f800c55" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 36379 } } peers { permanent_uuid: "2f7b8abb988b4425b0fcbd4226a3af46" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36645 } } peers { permanent_uuid: "1c2156e9807e4e9facd27fb69f3b0ba4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 34257 } }
I20260810 00:56:15.683727 5944 raft_consensus.cc:374] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:15.683848 5944 raft_consensus.cc:729] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 909d3151e0424fc49d26b96d6f800c55, State: Initialized, Role: FOLLOWER
I20260810 00:56:15.684365 5944 consensus_queue.cc:260] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909d3151e0424fc49d26b96d6f800c55" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 36379 } } peers { permanent_uuid: "2f7b8abb988b4425b0fcbd4226a3af46" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36645 } } peers { permanent_uuid: "1c2156e9807e4e9facd27fb69f3b0ba4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 34257 } }
I20260810 00:56:15.685305 5916 heartbeater.cc:503] Master 127.10.97.126:36253 was elected leader, sending a full tablet report...
I20260810 00:56:15.686003 5944 ts_tablet_manager.cc:1428] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
I20260810 00:56:15.692271 5945 raft_consensus.cc:348] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909d3151e0424fc49d26b96d6f800c55" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 36379 } } peers { permanent_uuid: "2f7b8abb988b4425b0fcbd4226a3af46" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36645 } } peers { permanent_uuid: "1c2156e9807e4e9facd27fb69f3b0ba4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 34257 } }
I20260810 00:56:15.692271 5950 raft_consensus.cc:348] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909d3151e0424fc49d26b96d6f800c55" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 36379 } } peers { permanent_uuid: "2f7b8abb988b4425b0fcbd4226a3af46" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36645 } } peers { permanent_uuid: "1c2156e9807e4e9facd27fb69f3b0ba4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 34257 } }
I20260810 00:56:15.692677 5950 raft_consensus.cc:374] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:15.692667 5945 raft_consensus.cc:374] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:15.692772 5945 raft_consensus.cc:729] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c2156e9807e4e9facd27fb69f3b0ba4, State: Initialized, Role: FOLLOWER
I20260810 00:56:15.692772 5950 raft_consensus.cc:729] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f7b8abb988b4425b0fcbd4226a3af46, State: Initialized, Role: FOLLOWER
I20260810 00:56:15.693312 5950 consensus_queue.cc:260] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909d3151e0424fc49d26b96d6f800c55" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 36379 } } peers { permanent_uuid: "2f7b8abb988b4425b0fcbd4226a3af46" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36645 } } peers { permanent_uuid: "1c2156e9807e4e9facd27fb69f3b0ba4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 34257 } }
I20260810 00:56:15.693312 5945 consensus_queue.cc:260] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909d3151e0424fc49d26b96d6f800c55" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 36379 } } peers { permanent_uuid: "2f7b8abb988b4425b0fcbd4226a3af46" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36645 } } peers { permanent_uuid: "1c2156e9807e4e9facd27fb69f3b0ba4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 34257 } }
I20260810 00:56:15.694289 5539 heartbeater.cc:503] Master 127.10.97.126:36253 was elected leader, sending a full tablet report...
I20260810 00:56:15.694713 5730 heartbeater.cc:503] Master 127.10.97.126:36253 was elected leader, sending a full tablet report...
I20260810 00:56:15.695251 5945 ts_tablet_manager.cc:1428] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260810 00:56:15.696357 5950 ts_tablet_manager.cc:1428] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260810 00:56:15.702752 5540 tablet.cc:2369] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:56:15.825459 5917 tablet.cc:2369] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:56:15.858705 5953 raft_consensus.cc:482] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:56:15.859038 5953 raft_consensus.cc:504] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909d3151e0424fc49d26b96d6f800c55" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 36379 } } peers { permanent_uuid: "2f7b8abb988b4425b0fcbd4226a3af46" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36645 } } peers { permanent_uuid: "1c2156e9807e4e9facd27fb69f3b0ba4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 34257 } }
I20260810 00:56:15.860765 5953 leader_election.cc:288] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 909d3151e0424fc49d26b96d6f800c55 (127.10.97.67:36379), 2f7b8abb988b4425b0fcbd4226a3af46 (127.10.97.65:36645)
I20260810 00:56:15.866961 5936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:15.861504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:51713 (local address 127.10.97.67:36379)
0810 00:56:15.861686 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:15.861692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:15.861784 (+ 92us) server_negotiation.cc:407] Connection header received
0810 00:56:15.861954 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:15.861959 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:15.862039 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:15.862155 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:15.862888 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.863648 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.865352 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.866121 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.866282 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:15.866353 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:15.866620 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 00:56:15.866744 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 00:56:15.867010 5955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:15.861093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:36379 (local address 127.10.97.66:51713)
0810 00:56:15.861691 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:15.861709 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:15.861830 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:15.862173 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:15.862326 (+ 153us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:15.862354 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:15.862735 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:15.862745 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.863779 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.863783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:15.865163 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:15.865176 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.866312 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.866317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:15.866469 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:15.866493 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 00:56:15.866572 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 00:56:15.866696 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":474,"threads_started":1}
I20260810 00:56:15.867540 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:15.861987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:36645 (local address 127.10.97.66:52293)
0810 00:56:15.862761 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:15.862779 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:15.862896 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:15.863213 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:15.863217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:15.863242 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:15.863541 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:15.863550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.864516 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.864520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:15.865820 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:15.865833 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.866919 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.866926 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:15.867097 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:15.867141 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 00:56:15.867222 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 00:56:15.867333 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":479,"threads_started":1}
I20260810 00:56:15.867638 5943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:15.862140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:52293 (local address 127.10.97.65:36645)
0810 00:56:15.862302 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:15.862306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:15.862871 (+ 565us) server_negotiation.cc:407] Connection header received
0810 00:56:15.863016 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:15.863020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:15.863090 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:15.863161 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:15.863688 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.864387 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.865991 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.866735 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.866890 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:15.866956 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:15.867265 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 00:56:15.867394 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 00:56:15.868135 5855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88238dfff9ef41fca85c5d1bc14f2de4" candidate_uuid: "1c2156e9807e4e9facd27fb69f3b0ba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "909d3151e0424fc49d26b96d6f800c55" is_pre_election: true
I20260810 00:56:15.868562 5472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88238dfff9ef41fca85c5d1bc14f2de4" candidate_uuid: "1c2156e9807e4e9facd27fb69f3b0ba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f7b8abb988b4425b0fcbd4226a3af46" is_pre_election: true
I20260810 00:56:15.868754 5855 raft_consensus.cc:2454] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c2156e9807e4e9facd27fb69f3b0ba4 in term 0.
I20260810 00:56:15.869086 5472 raft_consensus.cc:2454] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c2156e9807e4e9facd27fb69f3b0ba4 in term 0.
I20260810 00:56:15.869428 5567 leader_election.cc:302] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c2156e9807e4e9facd27fb69f3b0ba4, 909d3151e0424fc49d26b96d6f800c55; no voters:
I20260810 00:56:15.869812 5953 raft_consensus.cc:2781] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:56:15.869946 5953 raft_consensus.cc:482] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:56:15.870043 5953 raft_consensus.cc:3037] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:15.872217 5953 raft_consensus.cc:504] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909d3151e0424fc49d26b96d6f800c55" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 36379 } } peers { permanent_uuid: "2f7b8abb988b4425b0fcbd4226a3af46" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36645 } } peers { permanent_uuid: "1c2156e9807e4e9facd27fb69f3b0ba4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 34257 } }
I20260810 00:56:15.872726 5953 leader_election.cc:288] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [CANDIDATE]: Term 1 election: Requested vote from peers 909d3151e0424fc49d26b96d6f800c55 (127.10.97.67:36379), 2f7b8abb988b4425b0fcbd4226a3af46 (127.10.97.65:36645)
I20260810 00:56:15.873129 5855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88238dfff9ef41fca85c5d1bc14f2de4" candidate_uuid: "1c2156e9807e4e9facd27fb69f3b0ba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "909d3151e0424fc49d26b96d6f800c55"
I20260810 00:56:15.873180 5472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88238dfff9ef41fca85c5d1bc14f2de4" candidate_uuid: "1c2156e9807e4e9facd27fb69f3b0ba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f7b8abb988b4425b0fcbd4226a3af46"
I20260810 00:56:15.873296 5855 raft_consensus.cc:3037] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:15.873363 5472 raft_consensus.cc:3037] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:15.875358 5855 raft_consensus.cc:2454] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c2156e9807e4e9facd27fb69f3b0ba4 in term 1.
I20260810 00:56:15.875501 5472 raft_consensus.cc:2454] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c2156e9807e4e9facd27fb69f3b0ba4 in term 1.
I20260810 00:56:15.875694 5567 leader_election.cc:302] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c2156e9807e4e9facd27fb69f3b0ba4, 909d3151e0424fc49d26b96d6f800c55; no voters:
I20260810 00:56:15.875983 5953 raft_consensus.cc:2781] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:15.876368 5953 raft_consensus.cc:686] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [term 1 LEADER]: Becoming Leader. State: Replica: 1c2156e9807e4e9facd27fb69f3b0ba4, State: Running, Role: LEADER
I20260810 00:56:15.876787 5953 consensus_queue.cc:237] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909d3151e0424fc49d26b96d6f800c55" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 36379 } } peers { permanent_uuid: "2f7b8abb988b4425b0fcbd4226a3af46" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36645 } } peers { permanent_uuid: "1c2156e9807e4e9facd27fb69f3b0ba4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 34257 } }
I20260810 00:56:15.881162 5252 catalog_manager.cc:5495] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c2156e9807e4e9facd27fb69f3b0ba4 (127.10.97.66). New cstate: current_term: 1 leader_uuid: "1c2156e9807e4e9facd27fb69f3b0ba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909d3151e0424fc49d26b96d6f800c55" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 36379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f7b8abb988b4425b0fcbd4226a3af46" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c2156e9807e4e9facd27fb69f3b0ba4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 34257 } health_report { overall_health: HEALTHY } } }
W20260810 00:56:15.893029 5731 tablet.cc:2369] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:56:15.926478 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:15.921302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60070 (local address 127.10.97.66:34257)
0810 00:56:15.921469 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:15.921475 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:15.921493 (+ 18us) server_negotiation.cc:407] Connection header received
0810 00:56:15.921692 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:15.921695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:15.921772 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:15.921907 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:56:15.922284 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.922991 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.924474 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.924875 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.924980 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:15.925472 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:56:15.925951 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:56:15.926084 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:15.926183 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 00:56:15.926255 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 00:56:15.933640 5472 raft_consensus.cc:1261] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46 [term 1 FOLLOWER]: Refusing update from remote peer 1c2156e9807e4e9facd27fb69f3b0ba4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:56:15.933681 5855 raft_consensus.cc:1261] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55 [term 1 FOLLOWER]: Refusing update from remote peer 1c2156e9807e4e9facd27fb69f3b0ba4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:56:15.934475 5958 consensus_queue.cc:1035] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "909d3151e0424fc49d26b96d6f800c55" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 36379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:56:15.934782 5953 consensus_queue.cc:1035] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f7b8abb988b4425b0fcbd4226a3af46" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 36645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:56:15.944701 5963 mvcc.cc:205] Tried to move back new op lower bound from 7316780547815403520 to 7316780547594756096. Current Snapshot: MvccSnapshot[applied={T|T < 7316780547815403520}]
I20260810 00:56:15.950258 5964 mvcc.cc:205] Tried to move back new op lower bound from 7316780547815403520 to 7316780547594756096. Current Snapshot: MvccSnapshot[applied={T|T < 7316780547815403520}]
I20260810 00:56:15.958556 5251 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:56:15.958801 5251 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:56:15.962095 5251 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8534e373a454874ad4d9457260399ef: Sending DeleteTablet for 3 replicas of tablet 88238dfff9ef41fca85c5d1bc14f2de4
I20260810 00:56:15.963165 5827 tablet_service.cc:1514] Processing DeleteTablet for tablet 88238dfff9ef41fca85c5d1bc14f2de4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:56:15 UTC) from {username='slave'} at 127.0.0.1:58450
I20260810 00:56:15.963240 5439 tablet_service.cc:1514] Processing DeleteTablet for tablet 88238dfff9ef41fca85c5d1bc14f2de4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:56:15 UTC) from {username='slave'} at 127.0.0.1:47538
I20260810 00:56:15.963536 5633 tablet_service.cc:1514] Processing DeleteTablet for tablet 88238dfff9ef41fca85c5d1bc14f2de4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:56:15 UTC) from {username='slave'} at 127.0.0.1:60062
I20260810 00:56:15.964919 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 5341
I20260810 00:56:15.975565 5976 tablet_replica.cc:330] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46: stopping tablet replica
I20260810 00:56:15.976104 5976 raft_consensus.cc:2229] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:56:15.976625 5976 raft_consensus.cc:2258] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:56:15.978663 5976 ts_tablet_manager.cc:1905] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:56:15.985584 5976 ts_tablet_manager.cc:1918] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:56:15.985800 5976 log.cc:1196] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/88238dfff9ef41fca85c5d1bc14f2de4
I20260810 00:56:15.986378 5976 ts_tablet_manager.cc:1939] T 88238dfff9ef41fca85c5d1bc14f2de4 P 2f7b8abb988b4425b0fcbd4226a3af46: Deleting consensus metadata
I20260810 00:56:15.988112 5232 catalog_manager.cc:4841] TS 2f7b8abb988b4425b0fcbd4226a3af46 (127.10.97.65:36645): tablet 88238dfff9ef41fca85c5d1bc14f2de4 (table test-table [id=9ff6715b58274cb38eeae5afaef06736]) successfully deleted
I20260810 00:56:15.990651 5977 tablet_replica.cc:330] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4: stopping tablet replica
I20260810 00:56:15.991143 5977 raft_consensus.cc:2229] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:56:15.991847 5977 raft_consensus.cc:2258] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:56:15.994081 5977 ts_tablet_manager.cc:1905] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:56:15.996148 5975 tablet_replica.cc:330] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55: stopping tablet replica
I20260810 00:56:15.996594 5975 raft_consensus.cc:2229] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:56:15.997165 5975 raft_consensus.cc:2258] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:56:15.998175 5943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:15.966341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47544 (local address 127.10.97.65:36645)
0810 00:56:15.966509 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:15.966515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:15.990122 (+ 23607us) server_negotiation.cc:407] Connection header received
0810 00:56:15.990495 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:15.990501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:15.990596 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:15.990689 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:15.991338 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.992405 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.994282 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:15.994862 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:15.994951 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:15.996519 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:15.996529 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:15.996545 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:15.997265 (+ 720us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:15.997320 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:15.997345 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:15.997446 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:15.997742 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 00:56:15.997845 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 00:56:15.999140 5529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:15.999558 5975 ts_tablet_manager.cc:1905] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:56:16.001907 5977 ts_tablet_manager.cc:1918] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:56:16.002069 5977 log.cc:1196] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/88238dfff9ef41fca85c5d1bc14f2de4
I20260810 00:56:16.002650 5977 ts_tablet_manager.cc:1939] T 88238dfff9ef41fca85c5d1bc14f2de4 P 1c2156e9807e4e9facd27fb69f3b0ba4: Deleting consensus metadata
I20260810 00:56:16.004309 5233 catalog_manager.cc:4841] TS 1c2156e9807e4e9facd27fb69f3b0ba4 (127.10.97.66:34257): tablet 88238dfff9ef41fca85c5d1bc14f2de4 (table test-table [id=9ff6715b58274cb38eeae5afaef06736]) successfully deleted
I20260810 00:56:16.007686 5975 ts_tablet_manager.cc:1918] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:56:16.007874 5975 log.cc:1196] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/88238dfff9ef41fca85c5d1bc14f2de4
I20260810 00:56:16.008365 5975 ts_tablet_manager.cc:1939] T 88238dfff9ef41fca85c5d1bc14f2de4 P 909d3151e0424fc49d26b96d6f800c55: Deleting consensus metadata
I20260810 00:56:16.009861 5232 catalog_manager.cc:4841] TS 909d3151e0424fc49d26b96d6f800c55 (127.10.97.67:36379): tablet 88238dfff9ef41fca85c5d1bc14f2de4 (table test-table [id=9ff6715b58274cb38eeae5afaef06736]) successfully deleted
I20260810 00:56:16.092861 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 5341
I20260810 00:56:16.122977 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 5544
I20260810 00:56:16.134138 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:16.123722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60086 (local address 127.10.97.66:34257)
0810 00:56:16.123921 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:16.123925 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:16.123946 (+ 21us) server_negotiation.cc:407] Connection header received
0810 00:56:16.124145 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:16.124149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:16.124239 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:16.124323 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:16.124784 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:16.125611 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:16.127478 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:16.128091 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:16.128191 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:16.132777 (+ 4586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:16.132786 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:16.132802 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:16.133411 (+ 609us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:16.133462 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:16.133481 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:16.133558 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:16.133718 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 00:56:16.133798 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 00:56:16.134826 5721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:16.256359 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 5544
I20260810 00:56:16.277382 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 5733
I20260810 00:56:16.283483 5936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:16.278069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58454 (local address 127.10.97.67:36379)
0810 00:56:16.278232 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:16.278236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:16.278256 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:56:16.278411 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:16.278415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:16.278506 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:16.278585 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:16.278908 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:16.279612 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:16.280858 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:16.281250 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:16.281340 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:16.282236 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:16.282246 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:16.282260 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:16.282873 (+ 613us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:16.282923 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:16.282943 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:16.283035 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:16.283152 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 00:56:16.283236 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 00:56:16.284130 5909 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:16.369096 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 5733
I20260810 00:56:16.392865 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 5210
I20260810 00:56:16.393952 5294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:16.497535 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 5210
2026-08-10T00:56:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2828 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:56:16 dist-test-slave-jlzn krb5kdc[5996](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:56:16 dist-test-slave-jlzn krb5kdc[5996](info): set up 2 sockets
Aug 10 00:56:16 dist-test-slave-jlzn krb5kdc[5996](info): commencing operation
krb5kdc: starting...
W20260810 00:56:17.695381 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.004s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:56:17 dist-test-slave-jlzn krb5kdc[5996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:56:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:56:17.932126 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:39571
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:33173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:39571
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:18.245959 6021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:18.246234 6021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:18.246376 6021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:18.253288 6021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:18.253391 6021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:18.253468 6021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:18.253538 6021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:18.253597 6021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:18.262758 6021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:39571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:39571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:18.264097 6021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:18.265435 6021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:18.276597 6027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:18.276505 6033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:18.276554 6029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:18.277613 6021 server_base.cc:1034] running on GCE node
I20260810 00:56:18.278267 6021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:18.279484 6021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:18.280680 6021 hybrid_clock.cc:648] HybridClock initialized: now 1786323378280638 us; error 30 us; skew 500 ppm
I20260810 00:56:18.281000 6021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:18 dist-test-slave-jlzn krb5kdc[5996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323378, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:18.283891 6021 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:56:18.285182 6021 webserver.cc:460] Webserver started at http://127.10.97.126:43101/ using document root <none> and password file <none>
I20260810 00:56:18.285838 6021 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:18.285959 6021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:18.286226 6021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:18.288997 6021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "a4669b0151104b6fae803a2df8d50721"
format_stamp: "Formatted at 2026-08-10 00:56:18 on dist-test-slave-jlzn"
server_key: "9ab40164cdb6b75be82445ee0fdc0cc2"
server_key_iv: "b6fb0af3ec24d8cbede0f20bc25ab13e"
server_key_version: "encryptionkey@0"
I20260810 00:56:18.289688 6021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "a4669b0151104b6fae803a2df8d50721"
format_stamp: "Formatted at 2026-08-10 00:56:18 on dist-test-slave-jlzn"
server_key: "9ab40164cdb6b75be82445ee0fdc0cc2"
server_key_iv: "b6fb0af3ec24d8cbede0f20bc25ab13e"
server_key_version: "encryptionkey@0"
I20260810 00:56:18.294349 6021 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 00:56:18.297487 6043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:18.298651 6021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:56:18.298806 6021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "a4669b0151104b6fae803a2df8d50721"
format_stamp: "Formatted at 2026-08-10 00:56:18 on dist-test-slave-jlzn"
server_key: "9ab40164cdb6b75be82445ee0fdc0cc2"
server_key_iv: "b6fb0af3ec24d8cbede0f20bc25ab13e"
server_key_version: "encryptionkey@0"
I20260810 00:56:18.298974 6021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:18.314667 6021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:18.318086 6021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:18.318321 6021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:18.334782 6021 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:39571
I20260810 00:56:18.334800 6112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:39571 every 8 connection(s)
I20260810 00:56:18.335978 6021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:18.339298 6116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:18.342010 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 6021
I20260810 00:56:18.342207 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:18.342751 10629 external_mini_cluster.cc:1428] Setting key b09e2b4ee79c9d71c20e6fc425f626e8
I20260810 00:56:18.350721 6116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721: Bootstrap starting.
I20260810 00:56:18.354238 6116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:18.355443 6116 log.cc:824] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721: Log is configured to *not* fsync() on all Append() calls
Aug 10 00:56:18 dist-test-slave-jlzn krb5kdc[5996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:56:18.358088 6116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721: No bootstrap required, opened a new log
I20260810 00:56:18.364555 6125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:18.344496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35480 (local address 127.10.97.126:39571)
0810 00:56:18.345228 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:18.345238 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:18.345275 (+ 37us) server_negotiation.cc:407] Connection header received
0810 00:56:18.346884 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:18.346903 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:18.347247 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:18.347545 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:18.349305 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:18.350536 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:18.352437 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:18.353067 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:18.353201 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:18.357852 (+ 4651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:18.357859 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:18.357872 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:18.357904 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:18.360796 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:18.361402 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:18.361409 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:18.361414 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:18.361575 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:18.361950 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:18.361954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:18.361959 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:18.362470 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:18.362775 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:18.363247 (+ 472us) server_negotiation.cc:299] Negotiation successful
0810 00:56:18.363604 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":337,"threads_started":1}
I20260810 00:56:18.365499 6116 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4669b0151104b6fae803a2df8d50721" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39571 } }
I20260810 00:56:18.365849 6116 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:18.365942 6116 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4669b0151104b6fae803a2df8d50721, State: Initialized, Role: FOLLOWER
I20260810 00:56:18.366479 6116 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4669b0151104b6fae803a2df8d50721" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39571 } }
I20260810 00:56:18.366700 6116 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:18.366844 6116 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:18.367003 6116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:18.369107 6116 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4669b0151104b6fae803a2df8d50721" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39571 } }
I20260810 00:56:18.369555 6116 leader_election.cc:302] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4669b0151104b6fae803a2df8d50721; no voters:
I20260810 00:56:18.370083 6116 leader_election.cc:288] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:18.370177 6128 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:18.370793 6128 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721 [term 1 LEADER]: Becoming Leader. State: Replica: a4669b0151104b6fae803a2df8d50721, State: Running, Role: LEADER
I20260810 00:56:18.371265 6128 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4669b0151104b6fae803a2df8d50721" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39571 } }
I20260810 00:56:18.371896 6116 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:18.373811 6133 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4669b0151104b6fae803a2df8d50721. Latest consensus state: current_term: 1 leader_uuid: "a4669b0151104b6fae803a2df8d50721" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4669b0151104b6fae803a2df8d50721" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39571 } } }
I20260810 00:56:18.373891 6129 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4669b0151104b6fae803a2df8d50721" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4669b0151104b6fae803a2df8d50721" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 39571 } } }
I20260810 00:56:18.374197 6133 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:18.374240 6129 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:18.375113 6142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:18.383112 6142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:18.393893 6142 catalog_manager.cc:1347] Generated new cluster ID: bc0f23006f294ee99a112f1d97113ce9
I20260810 00:56:18.393996 6142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:18.431916 6142 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:18.433139 6142 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:18.444509 6142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721: Generated new TSK 0
I20260810 00:56:18.445529 6142 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
I20260810 00:56:18.520766 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:39571
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:33173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:18.848835 6168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:18.849094 6168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:18.849193 6168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:18.856107 6168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:18.856204 6168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:18.856318 6168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:56:18.866647 6168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:39571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:18.868099 6168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:18.869647 6168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:18.882467 6177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:18.882499 6178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:18.882948 6181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:18.883384 6168 server_base.cc:1034] running on GCE node
I20260810 00:56:18.883984 6168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:18.884800 6168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:18.885998 6168 hybrid_clock.cc:648] HybridClock initialized: now 1786323378885953 us; error 37 us; skew 500 ppm
I20260810 00:56:18.886293 6168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:18 dist-test-slave-jlzn krb5kdc[5996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323378, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:18.889372 6168 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:56:18.890703 6168 webserver.cc:460] Webserver started at http://127.10.97.65:43227/ using document root <none> and password file <none>
I20260810 00:56:18.891317 6168 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:18.891438 6168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:18.891738 6168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:18.894474 6168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "ff13a9d9059f4f3fb37b677aacc14f60"
format_stamp: "Formatted at 2026-08-10 00:56:18 on dist-test-slave-jlzn"
server_key: "cb3bfb1711a43d4c7fbc33bdfaa496b7"
server_key_iv: "2e7779eb6c6b646c9bcb5d7d57e01090"
server_key_version: "encryptionkey@0"
I20260810 00:56:18.895201 6168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "ff13a9d9059f4f3fb37b677aacc14f60"
format_stamp: "Formatted at 2026-08-10 00:56:18 on dist-test-slave-jlzn"
server_key: "cb3bfb1711a43d4c7fbc33bdfaa496b7"
server_key_iv: "2e7779eb6c6b646c9bcb5d7d57e01090"
server_key_version: "encryptionkey@0"
I20260810 00:56:18.900053 6168 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260810 00:56:18.903288 6192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:18.904477 6168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:56:18.904624 6168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "ff13a9d9059f4f3fb37b677aacc14f60"
format_stamp: "Formatted at 2026-08-10 00:56:18 on dist-test-slave-jlzn"
server_key: "cb3bfb1711a43d4c7fbc33bdfaa496b7"
server_key_iv: "2e7779eb6c6b646c9bcb5d7d57e01090"
server_key_version: "encryptionkey@0"
I20260810 00:56:18.904799 6168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:18.926764 6168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:18.930246 6168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:18.930488 6168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:18.931352 6168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:56:18.932556 6168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:56:18.932636 6168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:18.932756 6168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:56:18.932811 6168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:18.961840 6168 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:38473
I20260810 00:56:18.961896 6358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:38473 every 8 connection(s)
I20260810 00:56:18.962972 6168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:56:18.965538 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 6168
I20260810 00:56:18.965736 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:56:18.966240 10629 external_mini_cluster.cc:1428] Setting key e111d13d3b8e176655961997d08ebc9d
Aug 10 00:56:18 dist-test-slave-jlzn krb5kdc[5996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323378, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:56:18.982606 6362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:18.965380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:59097 (local address 127.10.97.126:39571)
0810 00:56:18.965898 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:18.965904 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:18.966852 (+ 948us) server_negotiation.cc:407] Connection header received
0810 00:56:18.968678 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:18.968692 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:18.968772 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:18.968878 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:18.970260 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:18.971219 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:18.972844 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:18.973428 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:18.973556 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:18.976825 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:18.976831 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:18.976835 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:18.976871 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:18.979708 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:18.980408 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:18.980413 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:18.980416 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:18.980562 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:18.980943 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:18.980947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:18.980950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:18.981318 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:18.981429 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:18.982177 (+ 748us) server_negotiation.cc:299] Negotiation successful
0810 00:56:18.982346 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":237,"threads_started":1}
I20260810 00:56:18.983734 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:18.965889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39571 (local address 127.10.97.65:59097)
0810 00:56:18.966683 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:18.966726 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:18.968398 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:18.969068 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:18.969075 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:18.969558 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:56:18.970067 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:18.970090 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:18.971357 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:18.971361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:18.972707 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:18.972721 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:18.973633 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:18.973639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:18.973821 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:18.974396 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:56:18.974417 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:56:18.976612 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:56:18.979875 (+ 3263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:56:18.979881 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:56:18.979891 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:56:18.980287 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:56:18.980661 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:56:18.980664 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:56:18.980666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:56:18.980840 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:56:18.981434 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:56:18.981442 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:56:18.981809 (+ 367us) client_negotiation.cc:769] Sending connection context
0810 00:56:18.982103 (+ 294us) client_negotiation.cc:240] Negotiation successful
0810 00:56:18.982572 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":309,"threads_started":1}
I20260810 00:56:18.985188 6359 heartbeater.cc:347] Connected to a master server at 127.10.97.126:39571
I20260810 00:56:18.985450 6359 heartbeater.cc:464] Registering TS with master...
I20260810 00:56:18.986096 6359 heartbeater.cc:511] Master 127.10.97.126:39571 requested a full tablet report, sending...
I20260810 00:56:18.988063 6066 ts_manager.cc:194] Registered new tserver with Master: ff13a9d9059f4f3fb37b677aacc14f60 (127.10.97.65:38473)
I20260810 00:56:18.989440 6066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:59097
WARNING: no policy specified for kudu/127.10.97.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
I20260810 00:56:19.026568 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:39571
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:33173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:19.333642 6367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:19.333887 6367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:19.334013 6367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:19.340957 6367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:19.341068 6367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:19.341214 6367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:56:19.350543 6367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:39571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:19.351574 6367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:19.352746 6367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:19.364516 6374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:19.364545 6375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:19.365113 6377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:19.365415 6367 server_base.cc:1034] running on GCE node
I20260810 00:56:19.365955 6367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:19.366744 6367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:19.367887 6367 hybrid_clock.cc:648] HybridClock initialized: now 1786323379367842 us; error 33 us; skew 500 ppm
I20260810 00:56:19.368158 6367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:19 dist-test-slave-jlzn krb5kdc[5996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323379, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:19.370949 6367 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:56:19.372262 6367 webserver.cc:460] Webserver started at http://127.10.97.66:42119/ using document root <none> and password file <none>
I20260810 00:56:19.372854 6367 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:19.372963 6367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:19.373235 6367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:19.375883 6367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "6b50baa6b0f84609bf2c795b6716a1f4"
format_stamp: "Formatted at 2026-08-10 00:56:19 on dist-test-slave-jlzn"
server_key: "3dc857571ee6b882ecbef5a804b05781"
server_key_iv: "1a6724f4cb2aadcb910f9a864ad7e58d"
server_key_version: "encryptionkey@0"
I20260810 00:56:19.376565 6367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "6b50baa6b0f84609bf2c795b6716a1f4"
format_stamp: "Formatted at 2026-08-10 00:56:19 on dist-test-slave-jlzn"
server_key: "3dc857571ee6b882ecbef5a804b05781"
server_key_iv: "1a6724f4cb2aadcb910f9a864ad7e58d"
server_key_version: "encryptionkey@0"
I20260810 00:56:19.381203 6367 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:56:19.384358 6385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:19.385506 6367 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:56:19.385651 6367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "6b50baa6b0f84609bf2c795b6716a1f4"
format_stamp: "Formatted at 2026-08-10 00:56:19 on dist-test-slave-jlzn"
server_key: "3dc857571ee6b882ecbef5a804b05781"
server_key_iv: "1a6724f4cb2aadcb910f9a864ad7e58d"
server_key_version: "encryptionkey@0"
I20260810 00:56:19.385824 6367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:19.411648 6367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:19.415032 6367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:19.415273 6367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:19.416190 6367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:56:19.417384 6367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:56:19.417455 6367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:19.417564 6367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:56:19.417614 6367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:19.446027 6367 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:45537
I20260810 00:56:19.446172 6556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:45537 every 8 connection(s)
I20260810 00:56:19.447150 6367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:56:19.456485 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 6367
I20260810 00:56:19.456626 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:56:19.457029 10629 external_mini_cluster.cc:1428] Setting key 17e27d7d34cc92a8c694df822e9a7dab
Aug 10 00:56:19 dist-test-slave-jlzn krb5kdc[5996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323379, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:56:19.465478 6362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:19.449405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:41865 (local address 127.10.97.126:39571)
0810 00:56:19.449580 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:19.449584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:19.450616 (+ 1032us) server_negotiation.cc:407] Connection header received
0810 00:56:19.452429 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:19.452435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:19.452513 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:19.452599 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:19.453989 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:19.454687 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:19.456197 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:19.456638 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:19.456761 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:19.460117 (+ 3356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:19.460122 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:19.460126 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:19.460159 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:19.462516 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:19.463250 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:19.463254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:19.463257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:19.463385 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:19.463791 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:19.463795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:19.463797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:19.464079 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:19.464212 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:19.464929 (+ 717us) server_negotiation.cc:299] Negotiation successful
0810 00:56:19.465159 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 00:56:19.466285 6561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:19.449819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39571 (local address 127.10.97.66:41865)
0810 00:56:19.450468 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:19.450510 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:19.452169 (+ 1659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:19.452828 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:19.452835 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:19.453307 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:56:19.453829 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:19.453846 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:19.454824 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:19.454828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:19.456056 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:19.456074 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:19.456859 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:19.456867 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:19.457069 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:19.457720 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:56:19.457748 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:56:19.459918 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:56:19.462644 (+ 2726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:56:19.462652 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:56:19.462679 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:56:19.463121 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:56:19.463491 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:56:19.463495 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:56:19.463499 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:56:19.463678 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:56:19.464205 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:56:19.464212 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:56:19.464559 (+ 347us) client_negotiation.cc:769] Sending connection context
0810 00:56:19.464777 (+ 218us) client_negotiation.cc:240] Negotiation successful
0810 00:56:19.465234 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":258,"threads_started":1}
I20260810 00:56:19.467511 6558 heartbeater.cc:347] Connected to a master server at 127.10.97.126:39571
I20260810 00:56:19.467772 6558 heartbeater.cc:464] Registering TS with master...
I20260810 00:56:19.468400 6558 heartbeater.cc:511] Master 127.10.97.126:39571 requested a full tablet report, sending...
I20260810 00:56:19.469753 6066 ts_manager.cc:194] Registered new tserver with Master: 6b50baa6b0f84609bf2c795b6716a1f4 (127.10.97.66:45537)
I20260810 00:56:19.470731 6066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:41865
WARNING: no policy specified for kudu/127.10.97.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
I20260810 00:56:19.517056 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:39571
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:33173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:19.831650 6566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:19.831938 6566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:19.832079 6566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:19.838953 6566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:19.839069 6566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:19.839221 6566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:56:19.848214 6566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:39571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:19.849296 6566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:19.850488 6566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:19.862519 6574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:19.862613 6579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:19.862634 6581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:19.863188 6566 server_base.cc:1034] running on GCE node
I20260810 00:56:19.863754 6566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:19.864576 6566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:19.865767 6566 hybrid_clock.cc:648] HybridClock initialized: now 1786323379865716 us; error 42 us; skew 500 ppm
I20260810 00:56:19.866044 6566 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:19 dist-test-slave-jlzn krb5kdc[5996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323379, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:19.868932 6566 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:56:19.870182 6566 webserver.cc:460] Webserver started at http://127.10.97.67:33127/ using document root <none> and password file <none>
I20260810 00:56:19.870772 6566 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:19.870889 6566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:19.871181 6566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:19.874094 6566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "75141af63b784c10b9eb508796d9cc89"
format_stamp: "Formatted at 2026-08-10 00:56:19 on dist-test-slave-jlzn"
server_key: "46b0af81bfbb4317805abd60ab782269"
server_key_iv: "966ead3966ab971aeb9d79a121ddd8d1"
server_key_version: "encryptionkey@0"
I20260810 00:56:19.874818 6566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "75141af63b784c10b9eb508796d9cc89"
format_stamp: "Formatted at 2026-08-10 00:56:19 on dist-test-slave-jlzn"
server_key: "46b0af81bfbb4317805abd60ab782269"
server_key_iv: "966ead3966ab971aeb9d79a121ddd8d1"
server_key_version: "encryptionkey@0"
I20260810 00:56:19.879511 6566 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260810 00:56:19.882848 6592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:19.884064 6566 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.003s
I20260810 00:56:19.884218 6566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "75141af63b784c10b9eb508796d9cc89"
format_stamp: "Formatted at 2026-08-10 00:56:19 on dist-test-slave-jlzn"
server_key: "46b0af81bfbb4317805abd60ab782269"
server_key_iv: "966ead3966ab971aeb9d79a121ddd8d1"
server_key_version: "encryptionkey@0"
I20260810 00:56:19.884403 6566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:19.897375 6566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:19.900818 6566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:19.901064 6566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:19.901890 6566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:56:19.903123 6566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:56:19.903213 6566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:19.903321 6566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:56:19.903378 6566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:19.932091 6566 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:34189
I20260810 00:56:19.932163 6755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:34189 every 8 connection(s)
I20260810 00:56:19.933213 6566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:56:19.938164 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 6566
I20260810 00:56:19.938329 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:56:19.938745 10629 external_mini_cluster.cc:1428] Setting key 6c9a85ab9591693daa70974a81520843
Aug 10 00:56:19 dist-test-slave-jlzn krb5kdc[5996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323379, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:56:19.951332 6362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:19.935447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:37195 (local address 127.10.97.126:39571)
0810 00:56:19.935604 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:19.935609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:19.936676 (+ 1067us) server_negotiation.cc:407] Connection header received
0810 00:56:19.938454 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:19.938459 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:19.938542 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:19.938629 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:19.940226 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:19.941282 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:19.942746 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:19.943139 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:19.943243 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:19.946310 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:19.946315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:19.946318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:19.946348 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:19.948596 (+ 2248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:19.949232 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:19.949236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:19.949238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:19.949366 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:19.949764 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:19.949767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:19.949769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:19.950018 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:19.950155 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:19.950769 (+ 614us) server_negotiation.cc:299] Negotiation successful
0810 00:56:19.950984 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 00:56:19.952180 6759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:19.935784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:39571 (local address 127.10.97.67:37195)
0810 00:56:19.936507 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:19.936557 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:19.938185 (+ 1628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:19.938861 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:19.938871 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:19.939387 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:56:19.940068 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:19.940088 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:19.941417 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:19.941421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:19.942615 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:19.942627 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:19.943341 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:19.943349 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:19.943572 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:19.944158 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:56:19.944179 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:56:19.946128 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:56:19.948723 (+ 2595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:56:19.948731 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:56:19.948744 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:56:19.949116 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:56:19.949465 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:56:19.949470 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:56:19.949474 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:56:19.949652 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:56:19.950145 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:56:19.950152 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:56:19.950420 (+ 268us) client_negotiation.cc:769] Sending connection context
0810 00:56:19.950630 (+ 210us) client_negotiation.cc:240] Negotiation successful
0810 00:56:19.951058 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":292,"threads_started":1}
I20260810 00:56:19.953394 6757 heartbeater.cc:347] Connected to a master server at 127.10.97.126:39571
I20260810 00:56:19.953647 6757 heartbeater.cc:464] Registering TS with master...
I20260810 00:56:19.954263 6757 heartbeater.cc:511] Master 127.10.97.126:39571 requested a full tablet report, sending...
I20260810 00:56:19.955507 6065 ts_manager.cc:194] Registered new tserver with Master: 75141af63b784c10b9eb508796d9cc89 (127.10.97.67:34189)
I20260810 00:56:19.956377 6065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:37195
I20260810 00:56:19.965292 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:56:19.980566 6362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:19.969171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35496 (local address 127.10.97.126:39571)
0810 00:56:19.969292 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:19.969296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:19.969760 (+ 464us) server_negotiation.cc:407] Connection header received
0810 00:56:19.969863 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:19.969866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:19.969917 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:19.969986 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:19.971301 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:19.972045 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:19.973360 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:19.973754 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:19.973842 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:19.976175 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:19.976179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:19.976183 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:19.976210 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:19.978332 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:19.978973 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:19.978977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:19.978979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:19.979093 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:19.979428 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:19.979431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:19.979433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:19.979728 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:19.979963 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:19.980217 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 00:56:19.980338 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 00:56:19.983888 6065 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:56:19.990411 6065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:56:19.992012 6359 heartbeater.cc:503] Master 127.10.97.126:39571 was elected leader, sending a full tablet report...
I20260810 00:56:20.011016 6780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:20.002013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45312 (local address 127.10.97.66:45537)
0810 00:56:20.003118 (+ 1105us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:20.003126 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:20.003155 (+ 29us) server_negotiation.cc:407] Connection header received
0810 00:56:20.003323 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:20.003331 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:20.003620 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:20.003838 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:20.004952 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.006122 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.008786 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.009872 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.010007 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:20.010208 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:20.010524 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 00:56:20.010699 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":941,"thread_start_us":1303,"threads_started":1}
I20260810 00:56:20.011142 6782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:20.002920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:38473 (local address 127.0.0.1:42018)
0810 00:56:20.003822 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:20.003838 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:20.003936 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:20.004969 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:20.004973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:20.004997 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:20.005350 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:20.005362 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.007123 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.007128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:20.009044 (+ 1916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:20.009062 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.010285 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.010292 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:20.010451 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:20.010481 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 00:56:20.010573 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 00:56:20.010741 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"mutex_wait_us":27,"thread_start_us":704,"threads_started":1}
I20260810 00:56:20.011718 6778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:20.001525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:45537 (local address 127.0.0.1:45312)
0810 00:56:20.002779 (+ 1254us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:20.002795 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:20.002912 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:20.004084 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:20.004092 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:20.004149 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:20.004740 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:20.004749 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.006250 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.006256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:20.008615 (+ 2359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:20.008630 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.010062 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.010068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:20.010270 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:20.010321 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 00:56:20.011254 (+ 933us) client_negotiation.cc:240] Negotiation successful
0810 00:56:20.011377 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1135,"mutex_wait_us":111,"spinlock_wait_cycles":1920,"thread_start_us":994,"threads_started":1}
I20260810 00:56:20.011853 6786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:20.003440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42018 (local address 127.10.97.65:38473)
0810 00:56:20.004470 (+ 1030us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:20.004478 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:20.004503 (+ 25us) server_negotiation.cc:407] Connection header received
0810 00:56:20.004584 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:20.004591 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:20.004794 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:20.004982 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:20.005523 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.006963 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.009194 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.010076 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.010200 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:20.010408 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:20.011354 (+ 946us) server_negotiation.cc:299] Negotiation successful
0810 00:56:20.011517 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":877,"thread_start_us":1174,"threads_started":1}
I20260810 00:56:20.013111 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:20.001299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:34189 (local address 127.0.0.1:50430)
0810 00:56:20.002274 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:20.002322 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:20.002526 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:20.004100 (+ 1574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:20.004105 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:20.004155 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:20.004735 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:20.004748 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.006666 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.006675 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:20.009064 (+ 2389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:20.009076 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.012448 (+ 3372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.012454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:20.012607 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:20.012638 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 00:56:20.012733 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 00:56:20.012854 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"mutex_wait_us":224,"spinlock_wait_cycles":4480,"thread_start_us":557,"threads_started":1}
I20260810 00:56:20.015496 6261 tablet_service.cc:1467] Processing CreateTablet for tablet 15d9e92ab1db4835ac6ddace7fe467f7 (DEFAULT_TABLE table=test-table [id=032624187f794037832aebbd0d9f9e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:56:20.015496 6455 tablet_service.cc:1467] Processing CreateTablet for tablet 15d9e92ab1db4835ac6ddace7fe467f7 (DEFAULT_TABLE table=test-table [id=032624187f794037832aebbd0d9f9e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:56:20.016957 6455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15d9e92ab1db4835ac6ddace7fe467f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:20.016965 6261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15d9e92ab1db4835ac6ddace7fe467f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:20.020092 6776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:20.001541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50430 (local address 127.10.97.67:34189)
0810 00:56:20.003518 (+ 1977us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:20.003525 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:20.003555 (+ 30us) server_negotiation.cc:407] Connection header received
0810 00:56:20.003642 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:20.003649 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:20.003896 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:20.004833 (+ 937us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:20.005103 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.006530 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.010490 (+ 3960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.012260 (+ 1770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.012383 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:20.012580 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:20.019585 (+ 7005us) server_negotiation.cc:299] Negotiation successful
0810 00:56:20.019778 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1746,"thread_start_us":1610,"threads_started":1}
I20260810 00:56:20.022766 6660 tablet_service.cc:1467] Processing CreateTablet for tablet 15d9e92ab1db4835ac6ddace7fe467f7 (DEFAULT_TABLE table=test-table [id=032624187f794037832aebbd0d9f9e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:56:20.023881 6660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15d9e92ab1db4835ac6ddace7fe467f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:20.030869 6788 tablet_bootstrap.cc:492] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60: Bootstrap starting.
I20260810 00:56:20.031621 6787 tablet_bootstrap.cc:492] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4: Bootstrap starting.
I20260810 00:56:20.035205 6788 tablet_bootstrap.cc:654] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:20.035249 6790 tablet_bootstrap.cc:492] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89: Bootstrap starting.
I20260810 00:56:20.036573 6788 log.cc:824] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:20.038524 6790 tablet_bootstrap.cc:654] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:20.038864 6787 tablet_bootstrap.cc:654] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:20.038949 6788 tablet_bootstrap.cc:492] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60: No bootstrap required, opened a new log
I20260810 00:56:20.039179 6788 ts_tablet_manager.cc:1397] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.005s
I20260810 00:56:20.039707 6790 log.cc:824] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:20.040002 6787 log.cc:824] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:20.042367 6790 tablet_bootstrap.cc:492] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89: No bootstrap required, opened a new log
I20260810 00:56:20.042573 6790 ts_tablet_manager.cc:1397] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260810 00:56:20.046448 6788 raft_consensus.cc:348] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75141af63b784c10b9eb508796d9cc89" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34189 } } peers { permanent_uuid: "6b50baa6b0f84609bf2c795b6716a1f4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 45537 } } peers { permanent_uuid: "ff13a9d9059f4f3fb37b677aacc14f60" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38473 } }
I20260810 00:56:20.046772 6788 raft_consensus.cc:374] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:20.046885 6788 raft_consensus.cc:729] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff13a9d9059f4f3fb37b677aacc14f60, State: Initialized, Role: FOLLOWER
I20260810 00:56:20.047443 6788 consensus_queue.cc:260] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75141af63b784c10b9eb508796d9cc89" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34189 } } peers { permanent_uuid: "6b50baa6b0f84609bf2c795b6716a1f4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 45537 } } peers { permanent_uuid: "ff13a9d9059f4f3fb37b677aacc14f60" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38473 } }
I20260810 00:56:20.048702 6787 tablet_bootstrap.cc:492] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4: No bootstrap required, opened a new log
I20260810 00:56:20.048949 6787 ts_tablet_manager.cc:1397] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.000s
I20260810 00:56:20.049746 6788 ts_tablet_manager.cc:1428] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60: Time spent starting tablet: real 0.010s user 0.008s sys 0.003s
I20260810 00:56:20.049940 6790 raft_consensus.cc:348] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75141af63b784c10b9eb508796d9cc89" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34189 } } peers { permanent_uuid: "6b50baa6b0f84609bf2c795b6716a1f4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 45537 } } peers { permanent_uuid: "ff13a9d9059f4f3fb37b677aacc14f60" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38473 } }
I20260810 00:56:20.050367 6790 raft_consensus.cc:374] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:20.050462 6790 raft_consensus.cc:729] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75141af63b784c10b9eb508796d9cc89, State: Initialized, Role: FOLLOWER
I20260810 00:56:20.050999 6790 consensus_queue.cc:260] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75141af63b784c10b9eb508796d9cc89" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34189 } } peers { permanent_uuid: "6b50baa6b0f84609bf2c795b6716a1f4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 45537 } } peers { permanent_uuid: "ff13a9d9059f4f3fb37b677aacc14f60" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38473 } }
I20260810 00:56:20.052075 6757 heartbeater.cc:503] Master 127.10.97.126:39571 was elected leader, sending a full tablet report...
I20260810 00:56:20.052763 6790 ts_tablet_manager.cc:1428] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260810 00:56:20.058209 6787 raft_consensus.cc:348] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75141af63b784c10b9eb508796d9cc89" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34189 } } peers { permanent_uuid: "6b50baa6b0f84609bf2c795b6716a1f4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 45537 } } peers { permanent_uuid: "ff13a9d9059f4f3fb37b677aacc14f60" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38473 } }
I20260810 00:56:20.058502 6787 raft_consensus.cc:374] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:20.058571 6787 raft_consensus.cc:729] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b50baa6b0f84609bf2c795b6716a1f4, State: Initialized, Role: FOLLOWER
I20260810 00:56:20.058965 6787 consensus_queue.cc:260] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75141af63b784c10b9eb508796d9cc89" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34189 } } peers { permanent_uuid: "6b50baa6b0f84609bf2c795b6716a1f4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 45537 } } peers { permanent_uuid: "ff13a9d9059f4f3fb37b677aacc14f60" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38473 } }
I20260810 00:56:20.059773 6558 heartbeater.cc:503] Master 127.10.97.126:39571 was elected leader, sending a full tablet report...
I20260810 00:56:20.060210 6787 ts_tablet_manager.cc:1428] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260810 00:56:20.127416 6798 raft_consensus.cc:482] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:56:20.127676 6798 raft_consensus.cc:504] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75141af63b784c10b9eb508796d9cc89" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34189 } } peers { permanent_uuid: "6b50baa6b0f84609bf2c795b6716a1f4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 45537 } } peers { permanent_uuid: "ff13a9d9059f4f3fb37b677aacc14f60" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38473 } }
I20260810 00:56:20.129251 6798 leader_election.cc:288] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b50baa6b0f84609bf2c795b6716a1f4 (127.10.97.66:45537), ff13a9d9059f4f3fb37b677aacc14f60 (127.10.97.65:38473)
I20260810 00:56:20.134899 6759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:20.129574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:45537 (local address 127.10.97.67:41681)
0810 00:56:20.129762 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:20.129778 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:20.129884 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:20.130233 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:20.130238 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:20.130268 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:20.130456 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:20.130464 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.131633 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.131637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:20.133297 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:20.133312 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.134383 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.134387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:20.134518 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:20.134542 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 00:56:20.134610 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 00:56:20.134702 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":1}
I20260810 00:56:20.135026 6780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:20.129552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:41681 (local address 127.10.97.66:45537)
0810 00:56:20.129767 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:20.129774 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:20.129824 (+ 50us) server_negotiation.cc:407] Connection header received
0810 00:56:20.130015 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:20.130021 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:20.130098 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:20.130195 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:20.130603 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.131485 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.133456 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.134250 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.134390 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:20.134460 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:20.134657 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 00:56:20.134781 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 00:56:20.135917 6485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15d9e92ab1db4835ac6ddace7fe467f7" candidate_uuid: "75141af63b784c10b9eb508796d9cc89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b50baa6b0f84609bf2c795b6716a1f4" is_pre_election: true
I20260810 00:56:20.136138 6786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:20.130119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:58909 (local address 127.10.97.65:38473)
0810 00:56:20.130306 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:20.130312 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:20.130817 (+ 505us) server_negotiation.cc:407] Connection header received
0810 00:56:20.130927 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:20.130932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:20.131003 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:20.131093 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:20.131506 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.132278 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.134209 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.135322 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.135476 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:20.135542 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:20.135892 (+ 350us) server_negotiation.cc:299] Negotiation successful
0810 00:56:20.135976 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 00:56:20.136330 6802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:20.129981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:38473 (local address 127.10.97.67:58909)
0810 00:56:20.130722 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:20.130735 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:20.130832 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:20.131096 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:20.131098 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:20.131113 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:20.131372 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:20.131379 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.132416 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.132422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:20.134058 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:20.134069 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.135516 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.135522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:20.135699 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:20.135731 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 00:56:20.135859 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 00:56:20.135981 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":23,"thread_start_us":480,"threads_started":1}
I20260810 00:56:20.136471 6485 raft_consensus.cc:2454] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75141af63b784c10b9eb508796d9cc89 in term 0.
I20260810 00:56:20.137135 6597 leader_election.cc:302] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b50baa6b0f84609bf2c795b6716a1f4, 75141af63b784c10b9eb508796d9cc89; no voters:
I20260810 00:56:20.137235 6284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15d9e92ab1db4835ac6ddace7fe467f7" candidate_uuid: "75141af63b784c10b9eb508796d9cc89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff13a9d9059f4f3fb37b677aacc14f60" is_pre_election: true
I20260810 00:56:20.137526 6798 raft_consensus.cc:2781] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:56:20.137674 6798 raft_consensus.cc:482] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:56:20.137768 6798 raft_consensus.cc:3037] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:20.137748 6284 raft_consensus.cc:2454] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75141af63b784c10b9eb508796d9cc89 in term 0.
I20260810 00:56:20.139971 6798 raft_consensus.cc:504] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75141af63b784c10b9eb508796d9cc89" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34189 } } peers { permanent_uuid: "6b50baa6b0f84609bf2c795b6716a1f4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 45537 } } peers { permanent_uuid: "ff13a9d9059f4f3fb37b677aacc14f60" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38473 } }
I20260810 00:56:20.140475 6798 leader_election.cc:288] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [CANDIDATE]: Term 1 election: Requested vote from peers 6b50baa6b0f84609bf2c795b6716a1f4 (127.10.97.66:45537), ff13a9d9059f4f3fb37b677aacc14f60 (127.10.97.65:38473)
I20260810 00:56:20.140832 6485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15d9e92ab1db4835ac6ddace7fe467f7" candidate_uuid: "75141af63b784c10b9eb508796d9cc89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b50baa6b0f84609bf2c795b6716a1f4"
I20260810 00:56:20.140892 6284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15d9e92ab1db4835ac6ddace7fe467f7" candidate_uuid: "75141af63b784c10b9eb508796d9cc89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff13a9d9059f4f3fb37b677aacc14f60"
I20260810 00:56:20.140991 6485 raft_consensus.cc:3037] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:20.141062 6284 raft_consensus.cc:3037] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:20.143177 6485 raft_consensus.cc:2454] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75141af63b784c10b9eb508796d9cc89 in term 1.
I20260810 00:56:20.143232 6284 raft_consensus.cc:2454] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75141af63b784c10b9eb508796d9cc89 in term 1.
I20260810 00:56:20.143510 6597 leader_election.cc:302] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b50baa6b0f84609bf2c795b6716a1f4, 75141af63b784c10b9eb508796d9cc89; no voters:
I20260810 00:56:20.143767 6798 raft_consensus.cc:2781] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:20.144191 6798 raft_consensus.cc:686] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [term 1 LEADER]: Becoming Leader. State: Replica: 75141af63b784c10b9eb508796d9cc89, State: Running, Role: LEADER
I20260810 00:56:20.144608 6798 consensus_queue.cc:237] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75141af63b784c10b9eb508796d9cc89" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34189 } } peers { permanent_uuid: "6b50baa6b0f84609bf2c795b6716a1f4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 45537 } } peers { permanent_uuid: "ff13a9d9059f4f3fb37b677aacc14f60" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38473 } }
I20260810 00:56:20.148849 6066 catalog_manager.cc:5495] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75141af63b784c10b9eb508796d9cc89 (127.10.97.67). New cstate: current_term: 1 leader_uuid: "75141af63b784c10b9eb508796d9cc89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75141af63b784c10b9eb508796d9cc89" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b50baa6b0f84609bf2c795b6716a1f4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 45537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff13a9d9059f4f3fb37b677aacc14f60" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38473 } health_report { overall_health: UNKNOWN } } }
I20260810 00:56:20.163075 6776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:20.158464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50444 (local address 127.10.97.67:34189)
0810 00:56:20.158640 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:20.158646 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:20.158766 (+ 120us) server_negotiation.cc:407] Connection header received
0810 00:56:20.158900 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:20.158903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:20.158973 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:20.159079 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:56:20.159367 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.160085 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.161385 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.161751 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.161834 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:20.162194 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:56:20.162602 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:56:20.162730 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:20.162808 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 00:56:20.162842 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 00:56:20.170049 6284 raft_consensus.cc:1261] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60 [term 1 FOLLOWER]: Refusing update from remote peer 75141af63b784c10b9eb508796d9cc89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:56:20.170084 6485 raft_consensus.cc:1261] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4 [term 1 FOLLOWER]: Refusing update from remote peer 75141af63b784c10b9eb508796d9cc89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:56:20.170818 6803 consensus_queue.cc:1035] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff13a9d9059f4f3fb37b677aacc14f60" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 38473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:56:20.171104 6798 consensus_queue.cc:1035] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b50baa6b0f84609bf2c795b6716a1f4" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 45537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:56:20.194247 6066 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:56:20.194478 6066 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:56:20.197574 6066 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a4669b0151104b6fae803a2df8d50721: Sending DeleteTablet for 3 replicas of tablet 15d9e92ab1db4835ac6ddace7fe467f7
I20260810 00:56:20.198622 6660 tablet_service.cc:1514] Processing DeleteTablet for tablet 15d9e92ab1db4835ac6ddace7fe467f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:56:20 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:50430
I20260810 00:56:20.198865 6455 tablet_service.cc:1514] Processing DeleteTablet for tablet 15d9e92ab1db4835ac6ddace7fe467f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:56:20 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:45312
I20260810 00:56:20.199256 6261 tablet_service.cc:1514] Processing DeleteTablet for tablet 15d9e92ab1db4835ac6ddace7fe467f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:56:20 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:42018
I20260810 00:56:20.203084 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 6168
I20260810 00:56:20.208622 6826 tablet_replica.cc:330] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60: stopping tablet replica
I20260810 00:56:20.209031 6826 raft_consensus.cc:2229] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:56:20.209487 6826 raft_consensus.cc:2258] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:56:20.210742 6823 tablet_replica.cc:330] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89: stopping tablet replica
I20260810 00:56:20.211042 6823 raft_consensus.cc:2229] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:56:20.211558 6823 raft_consensus.cc:2258] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:56:20.213363 6826 ts_tablet_manager.cc:1905] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:56:20.216383 6823 ts_tablet_manager.cc:1905] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 00:56:20 dist-test-slave-jlzn krb5kdc[5996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:56:20.220271 6826 ts_tablet_manager.cc:1918] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:56:20.220496 6826 log.cc:1196] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/15d9e92ab1db4835ac6ddace7fe467f7
I20260810 00:56:20.221117 6826 ts_tablet_manager.cc:1939] T 15d9e92ab1db4835ac6ddace7fe467f7 P ff13a9d9059f4f3fb37b677aacc14f60: Deleting consensus metadata
I20260810 00:56:20.222926 6045 catalog_manager.cc:4841] TS ff13a9d9059f4f3fb37b677aacc14f60 (127.10.97.65:38473): tablet 15d9e92ab1db4835ac6ddace7fe467f7 (table test-table [id=032624187f794037832aebbd0d9f9e0c]) successfully deleted
I20260810 00:56:20.223361 6823 ts_tablet_manager.cc:1918] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:56:20.223533 6823 log.cc:1196] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/15d9e92ab1db4835ac6ddace7fe467f7
I20260810 00:56:20.224648 6824 tablet_replica.cc:330] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4: stopping tablet replica
I20260810 00:56:20.225025 6824 raft_consensus.cc:2229] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:56:20.225275 6823 ts_tablet_manager.cc:1939] T 15d9e92ab1db4835ac6ddace7fe467f7 P 75141af63b784c10b9eb508796d9cc89: Deleting consensus metadata
I20260810 00:56:20.225464 6824 raft_consensus.cc:2258] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:56:20.226967 6048 catalog_manager.cc:4841] TS 75141af63b784c10b9eb508796d9cc89 (127.10.97.67:34189): tablet 15d9e92ab1db4835ac6ddace7fe467f7 (table test-table [id=032624187f794037832aebbd0d9f9e0c]) successfully deleted
I20260810 00:56:20.227389 6824 ts_tablet_manager.cc:1905] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:56:20.227900 6786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:20.204451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42032 (local address 127.10.97.65:38473)
0810 00:56:20.204619 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:20.204625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:20.207696 (+ 3071us) server_negotiation.cc:407] Connection header received
0810 00:56:20.208038 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:20.208044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:20.208123 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:20.208192 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:20.211918 (+ 3726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.213006 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.214775 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.215393 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.215490 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:20.220748 (+ 5258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:20.220766 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:20.220782 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:20.220826 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:20.224245 (+ 3419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:20.224905 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:20.224913 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:20.224917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:20.225075 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:20.225669 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:20.225674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:20.225677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:20.226118 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:20.226411 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:20.227373 (+ 962us) server_negotiation.cc:299] Negotiation successful
0810 00:56:20.227607 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 00:56:20.228503 6351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:20.233899 6824 ts_tablet_manager.cc:1918] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:56:20.234052 6824 log.cc:1196] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/15d9e92ab1db4835ac6ddace7fe467f7
I20260810 00:56:20.234558 6824 ts_tablet_manager.cc:1939] T 15d9e92ab1db4835ac6ddace7fe467f7 P 6b50baa6b0f84609bf2c795b6716a1f4: Deleting consensus metadata
I20260810 00:56:20.236168 6045 catalog_manager.cc:4841] TS 6b50baa6b0f84609bf2c795b6716a1f4 (127.10.97.66:45537): tablet 15d9e92ab1db4835ac6ddace7fe467f7 (table test-table [id=032624187f794037832aebbd0d9f9e0c]) successfully deleted
I20260810 00:56:20.316825 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 6168
I20260810 00:56:20.339798 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 6367
Aug 10 00:56:20 dist-test-slave-jlzn krb5kdc[5996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.66@KRBTEST.COM
I20260810 00:56:20.354972 6780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:20.340657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45316 (local address 127.10.97.66:45537)
0810 00:56:20.341090 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:20.341094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:20.341109 (+ 15us) server_negotiation.cc:407] Connection header received
0810 00:56:20.341161 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:20.341164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:20.341234 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:20.341320 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:20.342636 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.343392 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.344839 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.345243 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.345356 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:20.349476 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:20.349498 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:20.349513 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:20.349551 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:20.352285 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:20.352936 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:20.352943 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:20.352946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:20.353076 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:20.353438 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:20.353441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:20.353443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:20.353827 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:20.354112 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:20.354416 (+ 304us) server_negotiation.cc:299] Negotiation successful
0810 00:56:20.354637 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311}
I20260810 00:56:20.355624 6549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:20.441440 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 6367
I20260810 00:56:20.463766 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 6566
Aug 10 00:56:20 dist-test-slave-jlzn krb5kdc[5996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.67@KRBTEST.COM
I20260810 00:56:20.481487 6776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:20.464647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50452 (local address 127.10.97.67:34189)
0810 00:56:20.464810 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:20.464814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:20.464833 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:56:20.464973 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:20.464977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:20.465036 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:20.465131 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:20.466938 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.467743 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.469649 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:20.470114 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:20.470215 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:20.475631 (+ 5416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:20.475643 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:20.475657 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:20.475702 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:20.478604 (+ 2902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:20.479232 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:20.479239 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:20.479241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:20.479434 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:20.479938 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:20.479941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:20.479944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:20.480295 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:20.480603 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:20.480900 (+ 297us) server_negotiation.cc:299] Negotiation successful
0810 00:56:20.481106 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 00:56:20.482143 6749 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:20.598865 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 6566
I20260810 00:56:20.621685 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 6021
I20260810 00:56:20.622762 6107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:20.728335 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 6021
2026-08-10T00:56:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4235 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T00:56:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:20Z Disabled control of system clock
I20260810 00:56:20.813871 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:36599
--webserver_interface=127.10.97.126
--webserver_port=0
--builtin_ntp_servers=127.10.97.84:39463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:36599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 00:56:21.136118 6842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:21.136374 6842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:21.136492 6842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:21.143309 6842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:21.143402 6842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:21.143464 6842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:21.143540 6842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:21.143596 6842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:21.152241 6842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:36599
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:36599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:21.153339 6842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:21.155018 6842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:21.165817 6853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:21.165859 6852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:21.166285 6855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:21.166718 6842 server_base.cc:1034] running on GCE node
I20260810 00:56:21.167409 6842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:21.168685 6842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:21.169878 6842 hybrid_clock.cc:648] HybridClock initialized: now 1786323381169827 us; error 41 us; skew 500 ppm
I20260810 00:56:21.170176 6842 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:56:21.174695 6842 webserver.cc:460] Webserver started at http://127.10.97.126:36501/ using document root <none> and password file <none>
I20260810 00:56:21.175299 6842 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:21.175398 6842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:21.175698 6842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:21.178387 6842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "916d1ae82e684a01ada2a83c98702b17"
format_stamp: "Formatted at 2026-08-10 00:56:21 on dist-test-slave-jlzn"
server_key: "ac0040ff36e3da1db2da1aa9d063b8cc"
server_key_iv: "f081881fe0fc80d75c293c87b97a4e44"
server_key_version: "encryptionkey@0"
I20260810 00:56:21.179141 6842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "916d1ae82e684a01ada2a83c98702b17"
format_stamp: "Formatted at 2026-08-10 00:56:21 on dist-test-slave-jlzn"
server_key: "ac0040ff36e3da1db2da1aa9d063b8cc"
server_key_iv: "f081881fe0fc80d75c293c87b97a4e44"
server_key_version: "encryptionkey@0"
I20260810 00:56:21.183842 6842 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260810 00:56:21.186918 6862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:21.188136 6842 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:56:21.188292 6842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "916d1ae82e684a01ada2a83c98702b17"
format_stamp: "Formatted at 2026-08-10 00:56:21 on dist-test-slave-jlzn"
server_key: "ac0040ff36e3da1db2da1aa9d063b8cc"
server_key_iv: "f081881fe0fc80d75c293c87b97a4e44"
server_key_version: "encryptionkey@0"
I20260810 00:56:21.188474 6842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:21.220386 6842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:21.221093 6842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:21.221344 6842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:21.237880 6842 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:36599
I20260810 00:56:21.237903 6939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:36599 every 8 connection(s)
I20260810 00:56:21.239034 6842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:21.242443 6940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:21.244015 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 6842
I20260810 00:56:21.244200 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:21.244728 10629 external_mini_cluster.cc:1428] Setting key 862a6ad51cc9f03798f03083fa4992e6
I20260810 00:56:21.254413 6940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17: Bootstrap starting.
I20260810 00:56:21.256893 6944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:21.246569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40872 (local address 127.10.97.126:36599)
0810 00:56:21.247538 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:21.247548 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:21.247584 (+ 36us) server_negotiation.cc:407] Connection header received
0810 00:56:21.249128 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:21.249157 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:21.249569 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:21.249890 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:21.250386 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:21.251608 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:21.253001 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:21.253621 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:21.253784 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:21.254849 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:21.254857 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:21.254869 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:21.255340 (+ 471us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:21.255410 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:21.255437 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:21.255544 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:21.255752 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 00:56:21.256042 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":375,"threads_started":1}
I20260810 00:56:21.257409 6940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:21.258363 6940 log.cc:824] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:21.260872 6940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17: No bootstrap required, opened a new log
I20260810 00:56:21.267549 6940 raft_consensus.cc:348] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "916d1ae82e684a01ada2a83c98702b17" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 36599 } }
I20260810 00:56:21.267869 6940 raft_consensus.cc:374] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:21.267968 6940 raft_consensus.cc:729] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 916d1ae82e684a01ada2a83c98702b17, State: Initialized, Role: FOLLOWER
I20260810 00:56:21.268450 6940 consensus_queue.cc:260] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "916d1ae82e684a01ada2a83c98702b17" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 36599 } }
I20260810 00:56:21.268658 6940 raft_consensus.cc:388] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:21.268785 6940 raft_consensus.cc:482] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:21.268934 6940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:21.270874 6940 raft_consensus.cc:504] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "916d1ae82e684a01ada2a83c98702b17" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 36599 } }
I20260810 00:56:21.271330 6940 leader_election.cc:302] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 916d1ae82e684a01ada2a83c98702b17; no voters:
I20260810 00:56:21.271859 6940 leader_election.cc:288] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:21.271974 6950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:21.272816 6950 raft_consensus.cc:686] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17 [term 1 LEADER]: Becoming Leader. State: Replica: 916d1ae82e684a01ada2a83c98702b17, State: Running, Role: LEADER
I20260810 00:56:21.273360 6950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "916d1ae82e684a01ada2a83c98702b17" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 36599 } }
I20260810 00:56:21.273948 6940 sys_catalog.cc:564] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:21.275962 6951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "916d1ae82e684a01ada2a83c98702b17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "916d1ae82e684a01ada2a83c98702b17" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 36599 } } }
I20260810 00:56:21.276263 6952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 916d1ae82e684a01ada2a83c98702b17. Latest consensus state: current_term: 1 leader_uuid: "916d1ae82e684a01ada2a83c98702b17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "916d1ae82e684a01ada2a83c98702b17" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 36599 } } }
I20260810 00:56:21.276503 6951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:21.276549 6952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:21.284408 6964 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:21.288374 6964 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:21.295665 6964 catalog_manager.cc:1347] Generated new cluster ID: 814ab4bac5134c67a6778c869bace530
I20260810 00:56:21.295761 6964 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:21.314604 6964 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:21.316303 6964 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:21.322566 6964 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17: Generated new TSK 0
I20260810 00:56:21.323416 6964 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:56:21.334863 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:36599
--builtin_ntp_servers=127.10.97.84:39463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 00:56:21.655323 6983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:21.655555 6983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:21.655659 6983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:21.662803 6983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:21.662887 6983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:21.663012 6983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:56:21.671448 6983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:36599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:21.672528 6983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:21.674330 6983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:21.686337 6992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:21.686307 6996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:21.686391 6993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:21.687520 6983 server_base.cc:1034] running on GCE node
I20260810 00:56:21.688123 6983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:21.688910 6983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:21.690104 6983 hybrid_clock.cc:648] HybridClock initialized: now 1786323381690055 us; error 39 us; skew 500 ppm
I20260810 00:56:21.690383 6983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:56:21.692538 6983 webserver.cc:460] Webserver started at http://127.10.97.65:45423/ using document root <none> and password file <none>
I20260810 00:56:21.693158 6983 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:21.693275 6983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:21.693641 6983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:21.696334 6983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "663d19ab64984fcebd0bc3e1a23805e4"
format_stamp: "Formatted at 2026-08-10 00:56:21 on dist-test-slave-jlzn"
server_key: "ee807b33ac2275c73494f2f7576ee88d"
server_key_iv: "2f1b47911a0ff052c98cc3d3531da881"
server_key_version: "encryptionkey@0"
I20260810 00:56:21.697021 6983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "663d19ab64984fcebd0bc3e1a23805e4"
format_stamp: "Formatted at 2026-08-10 00:56:21 on dist-test-slave-jlzn"
server_key: "ee807b33ac2275c73494f2f7576ee88d"
server_key_iv: "2f1b47911a0ff052c98cc3d3531da881"
server_key_version: "encryptionkey@0"
I20260810 00:56:21.701501 6983 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260810 00:56:21.704522 7006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:21.705665 6983 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260810 00:56:21.705808 6983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "663d19ab64984fcebd0bc3e1a23805e4"
format_stamp: "Formatted at 2026-08-10 00:56:21 on dist-test-slave-jlzn"
server_key: "ee807b33ac2275c73494f2f7576ee88d"
server_key_iv: "2f1b47911a0ff052c98cc3d3531da881"
server_key_version: "encryptionkey@0"
I20260810 00:56:21.705987 6983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:21.718158 6983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:21.718814 6983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:21.719071 6983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:21.720045 6983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:56:21.721355 6983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:56:21.721451 6983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:21.721567 6983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:56:21.721643 6983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:21.750291 6983 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:44409
I20260810 00:56:21.750339 7173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:44409 every 8 connection(s)
I20260810 00:56:21.751427 6983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:56:21.753316 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 6983
I20260810 00:56:21.753464 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:56:21.753990 10629 external_mini_cluster.cc:1428] Setting key c4aa511986085fed1ebed8dd7d44c2a7
I20260810 00:56:21.756960 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:36599
--builtin_ntp_servers=127.10.97.84:39463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 00:56:21.763429 6944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:21.753786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:45483 (local address 127.10.97.126:36599)
0810 00:56:21.753995 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:21.754002 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:21.755483 (+ 1481us) server_negotiation.cc:407] Connection header received
0810 00:56:21.757303 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:21.757309 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:21.757383 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:21.757484 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:21.758411 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:21.759340 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:21.760849 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:21.761372 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:21.761484 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:21.762744 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:21.762750 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:21.762754 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:21.762774 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:21.762811 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:21.762826 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:21.762902 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:21.763074 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 00:56:21.763166 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 00:56:21.764271 7178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:21.754229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:36599 (local address 127.10.97.65:45483)
0810 00:56:21.755293 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:21.755353 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:21.757026 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:21.757642 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:21.757649 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:21.757725 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:56:21.758249 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:21.758266 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:21.759453 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:21.759457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:21.760709 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:21.760721 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:21.761582 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:21.761590 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:21.761856 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:21.762451 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:56:21.762454 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:56:21.762488 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:56:21.762554 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:56:21.762909 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:56:21.762915 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:56:21.762920 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 00:56:21.763039 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 00:56:21.763302 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":368,"threads_started":1}
I20260810 00:56:21.765532 7174 heartbeater.cc:347] Connected to a master server at 127.10.97.126:36599
I20260810 00:56:21.765827 7174 heartbeater.cc:464] Registering TS with master...
I20260810 00:56:21.766610 7174 heartbeater.cc:511] Master 127.10.97.126:36599 requested a full tablet report, sending...
I20260810 00:56:21.768568 6885 ts_manager.cc:194] Registered new tserver with Master: 663d19ab64984fcebd0bc3e1a23805e4 (127.10.97.65:44409)
I20260810 00:56:21.769975 6885 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.97.65:45483
W20260810 00:56:22.096917 7180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:22.097172 7180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:22.097287 7180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:22.104161 7180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:22.104254 7180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:22.104390 7180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:56:22.113211 7180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:36599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:22.114392 7180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:22.116050 7180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:22.127957 7189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:22.127985 7191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:22.128090 7195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:22.128934 7180 server_base.cc:1034] running on GCE node
I20260810 00:56:22.129487 7180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:22.130309 7180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:22.131536 7180 hybrid_clock.cc:648] HybridClock initialized: now 1786323382131470 us; error 27 us; skew 500 ppm
I20260810 00:56:22.131846 7180 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:56:22.133867 7180 webserver.cc:460] Webserver started at http://127.10.97.66:35003/ using document root <none> and password file <none>
I20260810 00:56:22.134487 7180 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:22.134598 7180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:22.134881 7180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:22.137583 7180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "361a74d26d6943c7a4fb3a1b1375b0cf"
format_stamp: "Formatted at 2026-08-10 00:56:22 on dist-test-slave-jlzn"
server_key: "80a97a52440f0df228f681a228d1044a"
server_key_iv: "1101a9c8ce7ba78d97daf8d6bc86befd"
server_key_version: "encryptionkey@0"
I20260810 00:56:22.138325 7180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "361a74d26d6943c7a4fb3a1b1375b0cf"
format_stamp: "Formatted at 2026-08-10 00:56:22 on dist-test-slave-jlzn"
server_key: "80a97a52440f0df228f681a228d1044a"
server_key_iv: "1101a9c8ce7ba78d97daf8d6bc86befd"
server_key_version: "encryptionkey@0"
I20260810 00:56:22.142967 7180 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 00:56:22.146152 7204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:22.147301 7180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:56:22.147449 7180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "361a74d26d6943c7a4fb3a1b1375b0cf"
format_stamp: "Formatted at 2026-08-10 00:56:22 on dist-test-slave-jlzn"
server_key: "80a97a52440f0df228f681a228d1044a"
server_key_iv: "1101a9c8ce7ba78d97daf8d6bc86befd"
server_key_version: "encryptionkey@0"
I20260810 00:56:22.147652 7180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:22.159914 7180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:22.160640 7180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:22.160908 7180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:22.161748 7180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:56:22.163049 7180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:56:22.163144 7180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:22.163250 7180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:56:22.163319 7180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:22.192586 7180 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:46683
I20260810 00:56:22.192636 7379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:46683 every 8 connection(s)
I20260810 00:56:22.193828 7180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:56:22.195410 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 7180
I20260810 00:56:22.195568 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:56:22.196125 10629 external_mini_cluster.cc:1428] Setting key aa8350786e2527d802dcab8802fb2e60
I20260810 00:56:22.199117 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:36599
--builtin_ntp_servers=127.10.97.84:39463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 00:56:22.206418 6944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:22.196471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:50089 (local address 127.10.97.126:36599)
0810 00:56:22.196663 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:22.196667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:22.198337 (+ 1670us) server_negotiation.cc:407] Connection header received
0810 00:56:22.200227 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:22.200233 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:22.200310 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:22.200410 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:22.201381 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.202335 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.203875 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.204384 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.204500 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:22.205703 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:22.205709 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:22.205714 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:22.205733 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:22.205768 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:22.205782 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:22.205862 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:22.206025 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 00:56:22.206119 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 00:56:22.207252 7383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:22.197109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:36599 (local address 127.10.97.66:50089)
0810 00:56:22.198168 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:22.198223 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:22.199906 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:22.200583 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:22.200590 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:22.200669 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:56:22.201224 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:22.201241 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.202453 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.202456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:22.203726 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:22.203738 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.204595 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.204601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:22.204793 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:22.205406 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:56:22.205408 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:56:22.205445 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:56:22.205514 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:56:22.205863 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:56:22.205869 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:56:22.205874 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 00:56:22.205994 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 00:56:22.206261 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":292,"threads_started":1}
I20260810 00:56:22.208541 7380 heartbeater.cc:347] Connected to a master server at 127.10.97.126:36599
I20260810 00:56:22.208804 7380 heartbeater.cc:464] Registering TS with master...
I20260810 00:56:22.209554 7380 heartbeater.cc:511] Master 127.10.97.126:36599 requested a full tablet report, sending...
I20260810 00:56:22.210906 6885 ts_manager.cc:194] Registered new tserver with Master: 361a74d26d6943c7a4fb3a1b1375b0cf (127.10.97.66:46683)
I20260810 00:56:22.211830 6885 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.97.66:50089
W20260810 00:56:22.519447 7384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:22.519719 7384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:22.519853 7384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:22.526687 7384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:22.526782 7384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:22.526925 7384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:56:22.535405 7384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:36599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:22.536506 7384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:22.538143 7384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:22.550583 7389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:22.550623 7395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:22.550652 7391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:22.551586 7384 server_base.cc:1034] running on GCE node
I20260810 00:56:22.552196 7384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:22.553020 7384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:22.554219 7384 hybrid_clock.cc:648] HybridClock initialized: now 1786323382554169 us; error 43 us; skew 500 ppm
I20260810 00:56:22.554499 7384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:56:22.556589 7384 webserver.cc:460] Webserver started at http://127.10.97.67:34035/ using document root <none> and password file <none>
I20260810 00:56:22.557200 7384 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:22.557304 7384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:22.557585 7384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:22.560237 7384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "7030db53a1574211ac8cc23b0434181d"
format_stamp: "Formatted at 2026-08-10 00:56:22 on dist-test-slave-jlzn"
server_key: "79c04be762943ddcffc1a8cc206b09c1"
server_key_iv: "982eb0077bb209df0d90c0ee30bccf27"
server_key_version: "encryptionkey@0"
I20260810 00:56:22.560972 7384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "7030db53a1574211ac8cc23b0434181d"
format_stamp: "Formatted at 2026-08-10 00:56:22 on dist-test-slave-jlzn"
server_key: "79c04be762943ddcffc1a8cc206b09c1"
server_key_iv: "982eb0077bb209df0d90c0ee30bccf27"
server_key_version: "encryptionkey@0"
I20260810 00:56:22.565627 7384 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:56:22.568765 7402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:22.569905 7384 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:56:22.570046 7384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "7030db53a1574211ac8cc23b0434181d"
format_stamp: "Formatted at 2026-08-10 00:56:22 on dist-test-slave-jlzn"
server_key: "79c04be762943ddcffc1a8cc206b09c1"
server_key_iv: "982eb0077bb209df0d90c0ee30bccf27"
server_key_version: "encryptionkey@0"
I20260810 00:56:22.570238 7384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:22.585374 7384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:22.586038 7384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:22.586308 7384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:22.587136 7384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:56:22.588476 7384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:56:22.588572 7384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:22.588678 7384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:56:22.588749 7384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:22.617369 7384 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:34861
I20260810 00:56:22.617424 7572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:34861 every 8 connection(s)
I20260810 00:56:22.618498 7384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:56:22.627116 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 7384
I20260810 00:56:22.627300 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:56:22.627722 10629 external_mini_cluster.cc:1428] Setting key 53ea61cd48be17f6d5eb82e60a4123eb
I20260810 00:56:22.630050 6944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:22.620720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:52921 (local address 127.10.97.126:36599)
0810 00:56:22.620923 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:22.620929 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:22.622168 (+ 1239us) server_negotiation.cc:407] Connection header received
0810 00:56:22.623840 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:22.623846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:22.623913 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:22.623991 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:22.624952 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.625656 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.627381 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.627792 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.627914 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:22.629259 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:22.629264 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:22.629267 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:22.629284 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:22.629316 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:22.629327 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:22.629401 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:22.629634 (+ 233us) server_negotiation.cc:299] Negotiation successful
0810 00:56:22.629732 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 00:56:22.630956 7577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:22.621220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:36599 (local address 127.10.97.67:52921)
0810 00:56:22.622035 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:22.622067 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:22.623578 (+ 1511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:22.624171 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:22.624178 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:22.624253 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:56:22.624792 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:22.624808 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.625795 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.625799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:22.627237 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:22.627255 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.628035 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.628044 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:22.628300 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:22.628967 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:56:22.628970 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:56:22.629006 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:56:22.629073 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:56:22.629417 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:56:22.629428 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:56:22.629435 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 00:56:22.629594 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 00:56:22.629932 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":257,"threads_started":1}
I20260810 00:56:22.632196 7573 heartbeater.cc:347] Connected to a master server at 127.10.97.126:36599
I20260810 00:56:22.632476 7573 heartbeater.cc:464] Registering TS with master...
I20260810 00:56:22.633206 7573 heartbeater.cc:511] Master 127.10.97.126:36599 requested a full tablet report, sending...
I20260810 00:56:22.634440 6885 ts_manager.cc:194] Registered new tserver with Master: 7030db53a1574211ac8cc23b0434181d (127.10.97.67:34861)
I20260810 00:56:22.635182 6885 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.97.67:52921
I20260810 00:56:22.642293 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:56:22.651479 6944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:22.646056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46502 (local address 127.10.97.126:36599)
0810 00:56:22.646234 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:22.646240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:22.646632 (+ 392us) server_negotiation.cc:407] Connection header received
0810 00:56:22.646743 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:22.646747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:22.646830 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:22.646906 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:22.647330 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.648072 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.649351 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.649712 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.649794 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:22.650828 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:22.650833 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:22.650836 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:22.650852 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:22.650882 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:22.650893 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:22.650991 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:22.651130 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 00:56:22.651178 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 00:56:22.654671 6885 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:56:22.661065 6885 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:56:22.695169 7595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:22.672139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:34861 (local address 127.0.0.1:55844)
0810 00:56:22.687920 (+ 15781us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:22.687936 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:22.688127 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:22.688821 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:22.688824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:22.688843 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:22.689269 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:22.689282 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.690724 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.690729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:22.692781 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:22.692793 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.694428 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.694433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:22.694561 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:22.694638 (+ 77us) client_negotiation.cc:769] Sending connection context
0810 00:56:22.694753 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 00:56:22.694920 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15649,"mutex_wait_us":10,"thread_start_us":15804,"threads_started":1}
I20260810 00:56:22.695869 7599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:22.673177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55844 (local address 127.10.97.67:34861)
0810 00:56:22.680687 (+ 7510us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:22.680695 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:22.688023 (+ 7328us) server_negotiation.cc:407] Connection header received
0810 00:56:22.688326 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:22.688333 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:22.688491 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:22.688598 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:22.689479 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.690580 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.692933 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.693916 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.694053 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:22.694327 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:22.695400 (+ 1073us) server_negotiation.cc:299] Negotiation successful
0810 00:56:22.695531 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7360,"thread_start_us":7254,"threads_started":1}
I20260810 00:56:22.696997 7593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:22.671380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:44409 (local address 127.0.0.1:53734)
0810 00:56:22.686662 (+ 15282us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:22.686708 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:22.686866 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:22.688572 (+ 1706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:22.688579 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:22.688613 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:22.689113 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:22.689137 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.691218 (+ 2081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.691227 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:22.693904 (+ 2677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:22.693920 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.696258 (+ 2338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.696264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:22.696442 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:22.696472 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 00:56:22.696549 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 00:56:22.696680 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15007,"mutex_wait_us":209,"thread_start_us":15660,"threads_started":1}
I20260810 00:56:22.697034 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:22.672758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33926 (local address 127.10.97.66:46683)
0810 00:56:22.680572 (+ 7814us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:22.680581 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:22.687924 (+ 7343us) server_negotiation.cc:407] Connection header received
0810 00:56:22.688398 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:22.688408 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:22.688610 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:22.688788 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:22.689653 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.691060 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.694214 (+ 3154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.695565 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.695668 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:22.695953 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:22.696542 (+ 589us) server_negotiation.cc:299] Negotiation successful
0810 00:56:22.696689 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7628,"thread_start_us":7504,"threads_started":1}
I20260810 00:56:22.697685 7594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:22.671644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53734 (local address 127.10.97.65:44409)
0810 00:56:22.687559 (+ 15915us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:22.687568 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:22.687597 (+ 29us) server_negotiation.cc:407] Connection header received
0810 00:56:22.687690 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:22.687697 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:22.687960 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:22.688158 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:22.689305 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.691057 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.694122 (+ 3065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.695776 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.695930 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:22.696141 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:22.697216 (+ 1075us) server_negotiation.cc:299] Negotiation successful
0810 00:56:22.697374 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15755,"thread_start_us":15655,"threads_started":1}
I20260810 00:56:22.697683 7596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:22.671696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:46683 (local address 127.0.0.1:33926)
0810 00:56:22.687798 (+ 16102us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:22.687834 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:22.688191 (+ 357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:22.689091 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:22.689095 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:22.689127 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:22.689498 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:22.689510 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.691222 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.691229 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:22.693993 (+ 2764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:22.694005 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.696113 (+ 2108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.696120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:22.696338 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:22.696371 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 00:56:22.697245 (+ 874us) client_negotiation.cc:240] Negotiation successful
0810 00:56:22.697369 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15967,"mutex_wait_us":167,"thread_start_us":16637,"threads_started":1}
I20260810 00:56:22.699764 7468 tablet_service.cc:1467] Processing CreateTablet for tablet b15b42846d5346b6b128ee260a9cb6b8 (DEFAULT_TABLE table=test-table [id=f816c4908fab4bc59ba2ccddc67e9144]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:56:22.700652 7077 tablet_service.cc:1467] Processing CreateTablet for tablet b15b42846d5346b6b128ee260a9cb6b8 (DEFAULT_TABLE table=test-table [id=f816c4908fab4bc59ba2ccddc67e9144]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:56:22.701329 7468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b15b42846d5346b6b128ee260a9cb6b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:22.701177 7283 tablet_service.cc:1467] Processing CreateTablet for tablet b15b42846d5346b6b128ee260a9cb6b8 (DEFAULT_TABLE table=test-table [id=f816c4908fab4bc59ba2ccddc67e9144]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:56:22.701694 7077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b15b42846d5346b6b128ee260a9cb6b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:22.702500 7283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b15b42846d5346b6b128ee260a9cb6b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:22.712750 7600 tablet_bootstrap.cc:492] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4: Bootstrap starting.
I20260810 00:56:22.715190 7601 tablet_bootstrap.cc:492] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d: Bootstrap starting.
I20260810 00:56:22.716279 7600 tablet_bootstrap.cc:654] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:22.717578 7600 log.cc:824] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:22.718461 7601 tablet_bootstrap.cc:654] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:22.719597 7601 log.cc:824] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:22.718917 7602 tablet_bootstrap.cc:492] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf: Bootstrap starting.
I20260810 00:56:22.720614 7600 tablet_bootstrap.cc:492] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4: No bootstrap required, opened a new log
I20260810 00:56:22.720854 7600 ts_tablet_manager.cc:1397] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260810 00:56:22.722373 7602 tablet_bootstrap.cc:654] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:22.722491 7601 tablet_bootstrap.cc:492] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d: No bootstrap required, opened a new log
I20260810 00:56:22.722734 7601 ts_tablet_manager.cc:1397] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260810 00:56:22.723352 7602 log.cc:824] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:22.726106 7602 tablet_bootstrap.cc:492] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf: No bootstrap required, opened a new log
I20260810 00:56:22.726342 7602 ts_tablet_manager.cc:1397] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260810 00:56:22.730715 7600 raft_consensus.cc:348] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663d19ab64984fcebd0bc3e1a23805e4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 44409 } } peers { permanent_uuid: "361a74d26d6943c7a4fb3a1b1375b0cf" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46683 } } peers { permanent_uuid: "7030db53a1574211ac8cc23b0434181d" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34861 } }
I20260810 00:56:22.731115 7600 raft_consensus.cc:374] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:22.731207 7600 raft_consensus.cc:729] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 663d19ab64984fcebd0bc3e1a23805e4, State: Initialized, Role: FOLLOWER
I20260810 00:56:22.731829 7600 consensus_queue.cc:260] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663d19ab64984fcebd0bc3e1a23805e4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 44409 } } peers { permanent_uuid: "361a74d26d6943c7a4fb3a1b1375b0cf" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46683 } } peers { permanent_uuid: "7030db53a1574211ac8cc23b0434181d" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34861 } }
I20260810 00:56:22.732867 7174 heartbeater.cc:503] Master 127.10.97.126:36599 was elected leader, sending a full tablet report...
I20260810 00:56:22.733021 7602 raft_consensus.cc:348] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663d19ab64984fcebd0bc3e1a23805e4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 44409 } } peers { permanent_uuid: "361a74d26d6943c7a4fb3a1b1375b0cf" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46683 } } peers { permanent_uuid: "7030db53a1574211ac8cc23b0434181d" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34861 } }
I20260810 00:56:22.733333 7602 raft_consensus.cc:374] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:22.733440 7602 raft_consensus.cc:729] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 361a74d26d6943c7a4fb3a1b1375b0cf, State: Initialized, Role: FOLLOWER
I20260810 00:56:22.733489 7600 ts_tablet_manager.cc:1428] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260810 00:56:22.732553 7601 raft_consensus.cc:348] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663d19ab64984fcebd0bc3e1a23805e4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 44409 } } peers { permanent_uuid: "361a74d26d6943c7a4fb3a1b1375b0cf" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46683 } } peers { permanent_uuid: "7030db53a1574211ac8cc23b0434181d" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34861 } }
I20260810 00:56:22.733810 7601 raft_consensus.cc:374] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:22.733916 7601 raft_consensus.cc:729] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7030db53a1574211ac8cc23b0434181d, State: Initialized, Role: FOLLOWER
I20260810 00:56:22.734122 7602 consensus_queue.cc:260] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663d19ab64984fcebd0bc3e1a23805e4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 44409 } } peers { permanent_uuid: "361a74d26d6943c7a4fb3a1b1375b0cf" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46683 } } peers { permanent_uuid: "7030db53a1574211ac8cc23b0434181d" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34861 } }
I20260810 00:56:22.734467 7601 consensus_queue.cc:260] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663d19ab64984fcebd0bc3e1a23805e4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 44409 } } peers { permanent_uuid: "361a74d26d6943c7a4fb3a1b1375b0cf" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46683 } } peers { permanent_uuid: "7030db53a1574211ac8cc23b0434181d" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34861 } }
I20260810 00:56:22.735925 7573 heartbeater.cc:503] Master 127.10.97.126:36599 was elected leader, sending a full tablet report...
I20260810 00:56:22.736333 7601 ts_tablet_manager.cc:1428] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260810 00:56:22.736487 7380 heartbeater.cc:503] Master 127.10.97.126:36599 was elected leader, sending a full tablet report...
I20260810 00:56:22.737474 7602 ts_tablet_manager.cc:1428] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf: Time spent starting tablet: real 0.011s user 0.000s sys 0.009s
I20260810 00:56:22.741957 7606 raft_consensus.cc:482] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:56:22.742192 7606 raft_consensus.cc:504] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663d19ab64984fcebd0bc3e1a23805e4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 44409 } } peers { permanent_uuid: "361a74d26d6943c7a4fb3a1b1375b0cf" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46683 } } peers { permanent_uuid: "7030db53a1574211ac8cc23b0434181d" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34861 } }
I20260810 00:56:22.743458 7606 leader_election.cc:288] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 361a74d26d6943c7a4fb3a1b1375b0cf (127.10.97.66:46683), 7030db53a1574211ac8cc23b0434181d (127.10.97.67:34861)
I20260810 00:56:22.751485 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:22.743999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:52575 (local address 127.10.97.66:46683)
0810 00:56:22.744166 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:22.744173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:22.744693 (+ 520us) server_negotiation.cc:407] Connection header received
0810 00:56:22.745047 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:22.745053 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:22.745134 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:22.745226 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:22.745799 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.746682 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.749278 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.750402 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.750576 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:22.750644 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:22.751149 (+ 505us) server_negotiation.cc:299] Negotiation successful
0810 00:56:22.751266 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 00:56:22.751616 7609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:22.743834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:46683 (local address 127.10.97.65:52575)
0810 00:56:22.744487 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:22.744504 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:22.744807 (+ 303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:22.745263 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:22.745268 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:22.745295 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:22.745672 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:22.745685 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.746809 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.746813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:22.749097 (+ 2284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:22.749112 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.750787 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.750794 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:22.750975 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:22.751007 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 00:56:22.751104 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 00:56:22.751250 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"mutex_wait_us":78,"thread_start_us":381,"threads_started":1}
I20260810 00:56:22.752022 7599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:22.744345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:53845 (local address 127.10.97.67:34861)
0810 00:56:22.744514 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:22.744519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:22.745368 (+ 849us) server_negotiation.cc:407] Connection header received
0810 00:56:22.745521 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:22.745524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:22.745573 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:22.745641 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:22.746221 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.747119 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.749491 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.750653 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.750803 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:22.750885 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:22.751274 (+ 389us) server_negotiation.cc:299] Negotiation successful
0810 00:56:22.751402 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 00:56:22.752295 7610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:22.744605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:34861 (local address 127.10.97.65:53845)
0810 00:56:22.745268 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:22.745285 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:22.745416 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:22.745718 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:22.745722 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:22.745749 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:22.746083 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:22.746092 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.747254 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.747259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:22.749341 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:22.749352 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.750847 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.750854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:22.751060 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:22.751099 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 00:56:22.751859 (+ 760us) client_negotiation.cc:240] Negotiation successful
0810 00:56:22.751952 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":20,"thread_start_us":342,"threads_started":1}
W20260810 00:56:22.753351 7177 tablet.cc:2369] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:56:22.753242 7496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15b42846d5346b6b128ee260a9cb6b8" candidate_uuid: "663d19ab64984fcebd0bc3e1a23805e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7030db53a1574211ac8cc23b0434181d" is_pre_election: true
I20260810 00:56:22.753748 7496 raft_consensus.cc:2454] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 663d19ab64984fcebd0bc3e1a23805e4 in term 0.
I20260810 00:56:22.754395 7009 leader_election.cc:302] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 663d19ab64984fcebd0bc3e1a23805e4, 7030db53a1574211ac8cc23b0434181d; no voters:
I20260810 00:56:22.754734 7606 raft_consensus.cc:2781] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:56:22.754853 7606 raft_consensus.cc:482] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:56:22.754954 7606 raft_consensus.cc:3037] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:22.757158 7606 raft_consensus.cc:504] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663d19ab64984fcebd0bc3e1a23805e4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 44409 } } peers { permanent_uuid: "361a74d26d6943c7a4fb3a1b1375b0cf" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46683 } } peers { permanent_uuid: "7030db53a1574211ac8cc23b0434181d" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34861 } }
I20260810 00:56:22.757683 7606 leader_election.cc:288] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [CANDIDATE]: Term 1 election: Requested vote from peers 361a74d26d6943c7a4fb3a1b1375b0cf (127.10.97.66:46683), 7030db53a1574211ac8cc23b0434181d (127.10.97.67:34861)
I20260810 00:56:22.758154 7496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15b42846d5346b6b128ee260a9cb6b8" candidate_uuid: "663d19ab64984fcebd0bc3e1a23805e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7030db53a1574211ac8cc23b0434181d"
I20260810 00:56:22.758339 7496 raft_consensus.cc:3037] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:22.760673 7306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15b42846d5346b6b128ee260a9cb6b8" candidate_uuid: "663d19ab64984fcebd0bc3e1a23805e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "361a74d26d6943c7a4fb3a1b1375b0cf"
I20260810 00:56:22.760713 7307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15b42846d5346b6b128ee260a9cb6b8" candidate_uuid: "663d19ab64984fcebd0bc3e1a23805e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "361a74d26d6943c7a4fb3a1b1375b0cf" is_pre_election: true
I20260810 00:56:22.761104 7306 raft_consensus.cc:3037] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:22.761358 7496 raft_consensus.cc:2454] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663d19ab64984fcebd0bc3e1a23805e4 in term 1.
I20260810 00:56:22.761762 7009 leader_election.cc:302] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 663d19ab64984fcebd0bc3e1a23805e4, 7030db53a1574211ac8cc23b0434181d; no voters:
I20260810 00:56:22.762028 7606 raft_consensus.cc:2781] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:22.762488 7606 raft_consensus.cc:686] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [term 1 LEADER]: Becoming Leader. State: Replica: 663d19ab64984fcebd0bc3e1a23805e4, State: Running, Role: LEADER
I20260810 00:56:22.763090 7606 consensus_queue.cc:237] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663d19ab64984fcebd0bc3e1a23805e4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 44409 } } peers { permanent_uuid: "361a74d26d6943c7a4fb3a1b1375b0cf" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46683 } } peers { permanent_uuid: "7030db53a1574211ac8cc23b0434181d" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34861 } }
I20260810 00:56:22.763883 7306 raft_consensus.cc:2454] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663d19ab64984fcebd0bc3e1a23805e4 in term 1.
I20260810 00:56:22.767206 6884 catalog_manager.cc:5495] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 663d19ab64984fcebd0bc3e1a23805e4 (127.10.97.65). New cstate: current_term: 1 leader_uuid: "663d19ab64984fcebd0bc3e1a23805e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663d19ab64984fcebd0bc3e1a23805e4" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 44409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "361a74d26d6943c7a4fb3a1b1375b0cf" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7030db53a1574211ac8cc23b0434181d" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34861 } health_report { overall_health: UNKNOWN } } }
I20260810 00:56:22.794947 7594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:22.789178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53736 (local address 127.10.97.65:44409)
0810 00:56:22.789370 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:22.789376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:22.789457 (+ 81us) server_negotiation.cc:407] Connection header received
0810 00:56:22.789604 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:22.789607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:22.789700 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:22.789812 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:56:22.790169 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.791023 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.792772 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.793286 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.793398 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:22.793943 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:56:22.794381 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:56:22.794536 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:22.794687 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 00:56:22.794725 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 00:56:22.802102 7306 raft_consensus.cc:1261] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf [term 1 FOLLOWER]: Refusing update from remote peer 663d19ab64984fcebd0bc3e1a23805e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:56:22.802239 7496 raft_consensus.cc:1261] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d [term 1 FOLLOWER]: Refusing update from remote peer 663d19ab64984fcebd0bc3e1a23805e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:56:22.802845 7611 consensus_queue.cc:1035] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "361a74d26d6943c7a4fb3a1b1375b0cf" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:56:22.803177 7606 consensus_queue.cc:1035] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7030db53a1574211ac8cc23b0434181d" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 34861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:56:22.813391 7615 mvcc.cc:205] Tried to move back new op lower bound from 7316780575949488128 to 7316780575801700352. Current Snapshot: MvccSnapshot[applied={T|T < 7316780575949488128}]
I20260810 00:56:22.820240 7617 mvcc.cc:205] Tried to move back new op lower bound from 7316780575949488128 to 7316780575801700352. Current Snapshot: MvccSnapshot[applied={T|T < 7316780575949488128}]
I20260810 00:56:22.826327 6884 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:56:22.826601 6884 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:56:22.830132 6884 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 916d1ae82e684a01ada2a83c98702b17: Sending DeleteTablet for 3 replicas of tablet b15b42846d5346b6b128ee260a9cb6b8
I20260810 00:56:22.831123 7077 tablet_service.cc:1514] Processing DeleteTablet for tablet b15b42846d5346b6b128ee260a9cb6b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:56:22 UTC) from {username='slave'} at 127.0.0.1:53734
I20260810 00:56:22.831287 7283 tablet_service.cc:1514] Processing DeleteTablet for tablet b15b42846d5346b6b128ee260a9cb6b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:56:22 UTC) from {username='slave'} at 127.0.0.1:33926
I20260810 00:56:22.831365 7468 tablet_service.cc:1514] Processing DeleteTablet for tablet b15b42846d5346b6b128ee260a9cb6b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:56:22 UTC) from {username='slave'} at 127.0.0.1:55844
I20260810 00:56:22.833508 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 6983
I20260810 00:56:22.843525 7630 tablet_replica.cc:330] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4: stopping tablet replica
I20260810 00:56:22.844086 7635 tablet_replica.cc:330] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d: stopping tablet replica
I20260810 00:56:22.844658 7635 raft_consensus.cc:2229] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:56:22.845152 7630 raft_consensus.cc:2229] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:56:22.845837 7630 raft_consensus.cc:2258] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:56:22.846390 7635 raft_consensus.cc:2258] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:56:22.849149 7635 ts_tablet_manager.cc:1905] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:56:22.849612 7630 ts_tablet_manager.cc:1905] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:56:22.856287 7632 tablet_replica.cc:330] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf: stopping tablet replica
I20260810 00:56:22.856662 7635 ts_tablet_manager.cc:1918] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:56:22.856662 7632 raft_consensus.cc:2229] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:56:22.856882 7635 log.cc:1196] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/b15b42846d5346b6b128ee260a9cb6b8
I20260810 00:56:22.857080 7630 ts_tablet_manager.cc:1918] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:56:22.857100 7632 raft_consensus.cc:2258] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:56:22.857246 7630 log.cc:1196] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/b15b42846d5346b6b128ee260a9cb6b8
I20260810 00:56:22.857609 7635 ts_tablet_manager.cc:1939] T b15b42846d5346b6b128ee260a9cb6b8 P 7030db53a1574211ac8cc23b0434181d: Deleting consensus metadata
I20260810 00:56:22.857798 7630 ts_tablet_manager.cc:1939] T b15b42846d5346b6b128ee260a9cb6b8 P 663d19ab64984fcebd0bc3e1a23805e4: Deleting consensus metadata
I20260810 00:56:22.859423 7594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:22.834755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53752 (local address 127.10.97.65:44409)
0810 00:56:22.835247 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:22.835253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:22.850496 (+ 15243us) server_negotiation.cc:407] Connection header received
0810 00:56:22.850744 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:22.850749 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:22.850843 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:22.850911 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:22.851608 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.852861 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.854699 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.855393 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.855489 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:22.857776 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:22.857791 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:22.857810 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:22.858557 (+ 747us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:22.858619 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:22.858644 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:22.858728 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:22.858970 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 00:56:22.859071 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362}
I20260810 00:56:22.859445 6869 catalog_manager.cc:4841] TS 663d19ab64984fcebd0bc3e1a23805e4 (127.10.97.65:44409): tablet b15b42846d5346b6b128ee260a9cb6b8 (table test-table [id=f816c4908fab4bc59ba2ccddc67e9144]) successfully deleted
I20260810 00:56:22.859472 6867 catalog_manager.cc:4841] TS 7030db53a1574211ac8cc23b0434181d (127.10.97.67:34861): tablet b15b42846d5346b6b128ee260a9cb6b8 (table test-table [id=f816c4908fab4bc59ba2ccddc67e9144]) successfully deleted
I20260810 00:56:22.860469 7167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:22.861523 7632 ts_tablet_manager.cc:1905] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:56:22.868439 7632 ts_tablet_manager.cc:1918] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:56:22.868583 7632 log.cc:1196] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/b15b42846d5346b6b128ee260a9cb6b8
I20260810 00:56:22.869050 7632 ts_tablet_manager.cc:1939] T b15b42846d5346b6b128ee260a9cb6b8 P 361a74d26d6943c7a4fb3a1b1375b0cf: Deleting consensus metadata
I20260810 00:56:22.870544 6864 catalog_manager.cc:4841] TS 361a74d26d6943c7a4fb3a1b1375b0cf (127.10.97.66:46683): tablet b15b42846d5346b6b128ee260a9cb6b8 (table test-table [id=f816c4908fab4bc59ba2ccddc67e9144]) successfully deleted
I20260810 00:56:22.959638 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 6983
I20260810 00:56:22.984825 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 7180
I20260810 00:56:22.993541 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:22.985532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33936 (local address 127.10.97.66:46683)
0810 00:56:22.985708 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:22.985715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:22.985779 (+ 64us) server_negotiation.cc:407] Connection header received
0810 00:56:22.985966 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:22.985970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:22.986079 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:22.986182 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:22.986576 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.987298 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.988697 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:22.989293 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:22.989410 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:22.990959 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:22.990972 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:22.991014 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:22.992144 (+ 1130us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:22.992242 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:22.992268 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:22.992964 (+ 696us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:22.993044 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 00:56:22.993153 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 00:56:22.995409 7371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:23.080564 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 7180
I20260810 00:56:23.108268 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 7384
I20260810 00:56:23.114809 7599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:23.109004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55856 (local address 127.10.97.67:34861)
0810 00:56:23.109192 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:23.109198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:23.109219 (+ 21us) server_negotiation.cc:407] Connection header received
0810 00:56:23.109385 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:23.109389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:23.109476 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:23.109560 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:23.109992 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:23.110711 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:23.111985 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:23.112408 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:23.112518 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:23.113496 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:23.113505 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:23.113521 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:23.114101 (+ 580us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:23.114163 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:23.114184 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:23.114281 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:23.114408 (+ 127us) server_negotiation.cc:299] Negotiation successful
0810 00:56:23.114512 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 00:56:23.115373 7565 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:23.200363 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 7384
I20260810 00:56:23.222867 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 6842
I20260810 00:56:23.223868 6932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:23.327255 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 6842
2026-08-10T00:56:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2597 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:56:23 dist-test-slave-jlzn krb5kdc[7646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:56:23 dist-test-slave-jlzn krb5kdc[7646](info): set up 2 sockets
Aug 10 00:56:23 dist-test-slave-jlzn krb5kdc[7646](info): commencing operation
krb5kdc: starting...
W20260810 00:56:24.520717 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.002s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:56:24 dist-test-slave-jlzn krb5kdc[7646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:56:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:56:24.764362 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:41297
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:43255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:41297
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:25.084070 7665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:25.084317 7665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:25.084409 7665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:25.091205 7665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:25.091298 7665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:25.091349 7665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:25.091405 7665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:25.091449 7665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:25.100937 7665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:43255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:41297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:41297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:25.102026 7665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:25.103197 7665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:25.114207 7677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:25.114267 7679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:25.114756 7676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:25.115022 7665 server_base.cc:1034] running on GCE node
I20260810 00:56:25.115687 7665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:25.116950 7665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:25.118144 7665 hybrid_clock.cc:648] HybridClock initialized: now 1786323385118099 us; error 31 us; skew 500 ppm
I20260810 00:56:25.118445 7665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:25 dist-test-slave-jlzn krb5kdc[7646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323385, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:25.121347 7665 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:56:25.122694 7665 webserver.cc:460] Webserver started at http://127.10.97.126:34443/ using document root <none> and password file <none>
I20260810 00:56:25.123265 7665 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:25.123363 7665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:25.123595 7665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:25.126690 7665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "ac1101e047bb4e58bca916302fab44df"
format_stamp: "Formatted at 2026-08-10 00:56:25 on dist-test-slave-jlzn"
server_key: "d20210b128159c148d42cc409d14ccc5"
server_key_iv: "b250a179ced229e8508ab0793bf20dcc"
server_key_version: "encryptionkey@0"
I20260810 00:56:25.127350 7665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "ac1101e047bb4e58bca916302fab44df"
format_stamp: "Formatted at 2026-08-10 00:56:25 on dist-test-slave-jlzn"
server_key: "d20210b128159c148d42cc409d14ccc5"
server_key_iv: "b250a179ced229e8508ab0793bf20dcc"
server_key_version: "encryptionkey@0"
I20260810 00:56:25.132045 7665 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260810 00:56:25.135113 7689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:25.136325 7665 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:56:25.136484 7665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "ac1101e047bb4e58bca916302fab44df"
format_stamp: "Formatted at 2026-08-10 00:56:25 on dist-test-slave-jlzn"
server_key: "d20210b128159c148d42cc409d14ccc5"
server_key_iv: "b250a179ced229e8508ab0793bf20dcc"
server_key_version: "encryptionkey@0"
I20260810 00:56:25.136667 7665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:25.150235 7665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:25.153699 7665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:25.153934 7665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:25.176482 7665 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:41297
I20260810 00:56:25.178368 7665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:25.176478 7760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:41297 every 8 connection(s)
I20260810 00:56:25.184535 7761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:25.185071 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 7665
I20260810 00:56:25.185248 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:25.185787 10629 external_mini_cluster.cc:1428] Setting key f8283a9b023fb63ea768e66ab73ee6ef
I20260810 00:56:25.198868 7761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df: Bootstrap starting.
Aug 10 00:56:25 dist-test-slave-jlzn krb5kdc[7646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:56:25.202353 7761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:25.203447 7761 log.cc:824] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:25.206845 7761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df: No bootstrap required, opened a new log
I20260810 00:56:25.211527 7766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:25.188308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42412 (local address 127.10.97.126:41297)
0810 00:56:25.189129 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:25.189144 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:25.189194 (+ 50us) server_negotiation.cc:407] Connection header received
0810 00:56:25.191091 (+ 1897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:25.191119 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:25.191607 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:25.192051 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:25.193362 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:25.195341 (+ 1979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:25.196972 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:25.197963 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:25.198135 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:25.203215 (+ 5080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:25.203227 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:25.203250 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:25.203299 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:25.206940 (+ 3641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:25.207754 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:25.207763 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:25.207770 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:25.207997 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:25.208407 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:25.208412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:25.208416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:25.209105 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:25.209403 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:25.209859 (+ 456us) server_negotiation.cc:299] Negotiation successful
0810 00:56:25.210287 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":341,"threads_started":1}
I20260810 00:56:25.216293 7761 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac1101e047bb4e58bca916302fab44df" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41297 } }
I20260810 00:56:25.216616 7761 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:25.216709 7761 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac1101e047bb4e58bca916302fab44df, State: Initialized, Role: FOLLOWER
I20260810 00:56:25.217231 7761 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac1101e047bb4e58bca916302fab44df" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41297 } }
I20260810 00:56:25.217453 7761 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:25.217577 7761 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:25.217724 7761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:25.219762 7761 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac1101e047bb4e58bca916302fab44df" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41297 } }
I20260810 00:56:25.220224 7761 leader_election.cc:302] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac1101e047bb4e58bca916302fab44df; no voters:
I20260810 00:56:25.220888 7761 leader_election.cc:288] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:25.220979 7772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:25.222954 7761 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:25.223287 7772 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df [term 1 LEADER]: Becoming Leader. State: Replica: ac1101e047bb4e58bca916302fab44df, State: Running, Role: LEADER
I20260810 00:56:25.223860 7772 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac1101e047bb4e58bca916302fab44df" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41297 } }
I20260810 00:56:25.226763 7773 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac1101e047bb4e58bca916302fab44df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac1101e047bb4e58bca916302fab44df" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41297 } } }
I20260810 00:56:25.226820 7776 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac1101e047bb4e58bca916302fab44df. Latest consensus state: current_term: 1 leader_uuid: "ac1101e047bb4e58bca916302fab44df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac1101e047bb4e58bca916302fab44df" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41297 } } }
I20260810 00:56:25.227152 7776 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:25.227149 7773 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:25.229087 7793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:25.233587 7793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:25.241395 7793 catalog_manager.cc:1347] Generated new cluster ID: 34974ee90a4e41edacce9f0673bde155
I20260810 00:56:25.241488 7793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:25.254074 7793 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:25.255760 7793 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:25.265893 7793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df: Generated new TSK 0
I20260810 00:56:25.266803 7793 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.97.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
Entry for principal kudu/127.10.97.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab.
I20260810 00:56:25.326807 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:41297
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:43255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:25.641757 7815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:25.642012 7815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:25.642144 7815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:25.648978 7815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:25.649081 7815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:25.649225 7815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:56:25.658275 7815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:43255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.65.keytab
--principal=kudu/127.10.97.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:41297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:25.659340 7815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:25.660547 7815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:25.672559 7823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:25.672480 7829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:25.672526 7826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:25.673344 7815 server_base.cc:1034] running on GCE node
I20260810 00:56:25.673871 7815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:25.674746 7815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:25.675891 7815 hybrid_clock.cc:648] HybridClock initialized: now 1786323385675852 us; error 29 us; skew 500 ppm
I20260810 00:56:25.676219 7815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:25 dist-test-slave-jlzn krb5kdc[7646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323385, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:25.679177 7815 init.cc:377] Logged in from keytab as kudu/127.10.97.65@KRBTEST.COM (short username kudu)
I20260810 00:56:25.680466 7815 webserver.cc:460] Webserver started at http://127.10.97.65:33605/ using document root <none> and password file <none>
I20260810 00:56:25.681041 7815 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:25.681149 7815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:25.681393 7815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:25.684010 7815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "93837d6279904ce29294a7d61e5406a9"
format_stamp: "Formatted at 2026-08-10 00:56:25 on dist-test-slave-jlzn"
server_key: "23f053093c214f9156af2d923c864a7c"
server_key_iv: "141c77edffb2fbb02edaf5c00e9cae86"
server_key_version: "encryptionkey@0"
I20260810 00:56:25.684705 7815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "93837d6279904ce29294a7d61e5406a9"
format_stamp: "Formatted at 2026-08-10 00:56:25 on dist-test-slave-jlzn"
server_key: "23f053093c214f9156af2d923c864a7c"
server_key_iv: "141c77edffb2fbb02edaf5c00e9cae86"
server_key_version: "encryptionkey@0"
I20260810 00:56:25.689296 7815 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:56:25.692409 7838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:25.693570 7815 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:56:25.693722 7815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "93837d6279904ce29294a7d61e5406a9"
format_stamp: "Formatted at 2026-08-10 00:56:25 on dist-test-slave-jlzn"
server_key: "23f053093c214f9156af2d923c864a7c"
server_key_iv: "141c77edffb2fbb02edaf5c00e9cae86"
server_key_version: "encryptionkey@0"
I20260810 00:56:25.693886 7815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:25.720180 7815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:25.724018 7815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:25.724267 7815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:25.725116 7815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:56:25.726364 7815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:56:25.726444 7815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:25.726553 7815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:56:25.726606 7815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:25.755415 7815 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:39021
I20260810 00:56:25.755465 8008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:39021 every 8 connection(s)
I20260810 00:56:25.756651 7815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:56:25.760190 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 7815
I20260810 00:56:25.760370 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:56:25.760846 10629 external_mini_cluster.cc:1428] Setting key 09da7923160b65bb7c8507b816ac6056
Aug 10 00:56:25 dist-test-slave-jlzn krb5kdc[7646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323385, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.65@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:56:25.776466 8013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:25.758948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:59553 (local address 127.10.97.126:41297)
0810 00:56:25.759388 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:25.759394 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:25.760373 (+ 979us) server_negotiation.cc:407] Connection header received
0810 00:56:25.762385 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:25.762389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:25.762457 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:25.762568 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:25.764083 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:25.765117 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:25.766701 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:25.767211 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:25.767332 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:25.770626 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:25.770632 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:25.770636 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:25.770673 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:25.773539 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:25.774276 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:25.774281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:25.774284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:25.774433 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:25.774834 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:25.774838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:25.774841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:25.775163 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:25.775297 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:25.775981 (+ 684us) server_negotiation.cc:299] Negotiation successful
0810 00:56:25.776196 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":202,"threads_started":1}
I20260810 00:56:25.777251 8014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:25.759474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:41297 (local address 127.10.97.65:59553)
0810 00:56:25.760190 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:25.760226 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:25.762116 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:25.762836 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:25.762847 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:25.763343 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:56:25.763915 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:25.763934 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:25.765272 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:25.765276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:25.766557 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:25.766571 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:25.767386 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:25.767391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:25.767568 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:25.768241 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:56:25.768270 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:56:25.770401 (+ 2131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:56:25.773667 (+ 3266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:56:25.773675 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:56:25.773687 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:56:25.774152 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:56:25.774528 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:56:25.774532 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:56:25.774535 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:56:25.774725 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:56:25.775273 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:56:25.775279 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:56:25.775602 (+ 323us) client_negotiation.cc:769] Sending connection context
0810 00:56:25.775855 (+ 253us) client_negotiation.cc:240] Negotiation successful
0810 00:56:25.776259 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":307,"threads_started":1}
I20260810 00:56:25.778581 8010 heartbeater.cc:347] Connected to a master server at 127.10.97.126:41297
I20260810 00:56:25.778874 8010 heartbeater.cc:464] Registering TS with master...
I20260810 00:56:25.779606 8010 heartbeater.cc:511] Master 127.10.97.126:41297 requested a full tablet report, sending...
I20260810 00:56:25.781721 7714 ts_manager.cc:194] Registered new tserver with Master: 93837d6279904ce29294a7d61e5406a9 (127.10.97.65:39021)
I20260810 00:56:25.783089 7714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.65@KRBTEST.COM'} at 127.10.97.65:59553
WARNING: no policy specified for kudu/127.10.97.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
Entry for principal kudu/127.10.97.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab.
I20260810 00:56:25.822919 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:41297
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:43255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:26.134759 8018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:26.134997 8018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:26.135120 8018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:26.141979 8018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:26.142074 8018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:26.142228 8018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:56:26.151208 8018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:43255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.66.keytab
--principal=kudu/127.10.97.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:41297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:26.152295 8018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:26.153446 8018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:26.165251 8027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:26.165246 8026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:26.165253 8030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:26.166159 8018 server_base.cc:1034] running on GCE node
I20260810 00:56:26.166710 8018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:26.167505 8018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:26.168696 8018 hybrid_clock.cc:648] HybridClock initialized: now 1786323386168649 us; error 42 us; skew 500 ppm
I20260810 00:56:26.168970 8018 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:26 dist-test-slave-jlzn krb5kdc[7646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323386, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:26.171826 8018 init.cc:377] Logged in from keytab as kudu/127.10.97.66@KRBTEST.COM (short username kudu)
I20260810 00:56:26.173137 8018 webserver.cc:460] Webserver started at http://127.10.97.66:33005/ using document root <none> and password file <none>
I20260810 00:56:26.173724 8018 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:26.173839 8018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:26.174121 8018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:26.176867 8018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "6636df8556f64f3982d53425f36ce966"
format_stamp: "Formatted at 2026-08-10 00:56:26 on dist-test-slave-jlzn"
server_key: "9821b5c4afcbe33810730183f58436e7"
server_key_iv: "fec6ba683ea0cfcace6ab21d88ed6f7f"
server_key_version: "encryptionkey@0"
I20260810 00:56:26.177564 8018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "6636df8556f64f3982d53425f36ce966"
format_stamp: "Formatted at 2026-08-10 00:56:26 on dist-test-slave-jlzn"
server_key: "9821b5c4afcbe33810730183f58436e7"
server_key_iv: "fec6ba683ea0cfcace6ab21d88ed6f7f"
server_key_version: "encryptionkey@0"
I20260810 00:56:26.182293 8018 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260810 00:56:26.185546 8038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:26.186771 8018 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 00:56:26.186914 8018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "6636df8556f64f3982d53425f36ce966"
format_stamp: "Formatted at 2026-08-10 00:56:26 on dist-test-slave-jlzn"
server_key: "9821b5c4afcbe33810730183f58436e7"
server_key_iv: "fec6ba683ea0cfcace6ab21d88ed6f7f"
server_key_version: "encryptionkey@0"
I20260810 00:56:26.187089 8018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:26.208231 8018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:26.211594 8018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:26.211895 8018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:26.212750 8018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:56:26.213963 8018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:56:26.214035 8018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:26.214130 8018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:56:26.214174 8018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:26.243151 8018 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:46735
I20260810 00:56:26.243209 8219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:46735 every 8 connection(s)
I20260810 00:56:26.244311 8018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:56:26.254236 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 8018
I20260810 00:56:26.254387 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:56:26.254869 10629 external_mini_cluster.cc:1428] Setting key b20b9fee85e1c9123a592ba9dfae1ccd
Aug 10 00:56:26 dist-test-slave-jlzn krb5kdc[7646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323386, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.66@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:56:26.262503 8013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:26.246634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:54697 (local address 127.10.97.126:41297)
0810 00:56:26.246808 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:26.246813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:26.247837 (+ 1024us) server_negotiation.cc:407] Connection header received
0810 00:56:26.249555 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:26.249561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:26.249645 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:26.249731 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:26.251100 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.251881 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.253339 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.253748 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.253858 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:26.257235 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:26.257240 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:26.257243 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:26.257273 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:26.259681 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:26.260350 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:26.260354 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:26.260357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:26.260486 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:26.260906 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:26.260909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:26.260912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:26.261173 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:26.261314 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:26.261947 (+ 633us) server_negotiation.cc:299] Negotiation successful
0810 00:56:26.262188 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 00:56:26.263273 8223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:26.247033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:41297 (local address 127.10.97.66:54697)
0810 00:56:26.247684 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:26.247719 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:26.249309 (+ 1590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:26.249960 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:26.249967 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:26.250440 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:56:26.250936 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:26.250954 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.252023 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.252027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:26.253201 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:26.253213 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.253985 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.253993 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:26.254205 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:26.254811 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:56:26.254838 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:56:26.257033 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:56:26.259821 (+ 2788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:56:26.259830 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:56:26.259844 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:56:26.260238 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:56:26.260591 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:56:26.260596 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:56:26.260600 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:56:26.260790 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:56:26.261294 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:56:26.261301 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:56:26.261584 (+ 283us) client_negotiation.cc:769] Sending connection context
0810 00:56:26.261814 (+ 230us) client_negotiation.cc:240] Negotiation successful
0810 00:56:26.262257 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":251,"threads_started":1}
I20260810 00:56:26.264588 8220 heartbeater.cc:347] Connected to a master server at 127.10.97.126:41297
I20260810 00:56:26.264835 8220 heartbeater.cc:464] Registering TS with master...
I20260810 00:56:26.265458 8220 heartbeater.cc:511] Master 127.10.97.126:41297 requested a full tablet report, sending...
I20260810 00:56:26.266789 7714 ts_manager.cc:194] Registered new tserver with Master: 6636df8556f64f3982d53425f36ce966 (127.10.97.66:46735)
I20260810 00:56:26.267732 7714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.66@KRBTEST.COM'} at 127.10.97.66:54697
WARNING: no policy specified for kudu/127.10.97.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
Entry for principal kudu/127.10.97.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab.
I20260810 00:56:26.312534 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:41297
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:43255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:26.622267 8227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:26.622524 8227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:26.622668 8227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:26.629632 8227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:26.629724 8227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:26.629832 8227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:56:26.638661 8227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:43255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.67.keytab
--principal=kudu/127.10.97.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:41297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:26.639701 8227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:26.640956 8227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:26.653121 8238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:26.653177 8233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:26.653316 8232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:26.653961 8227 server_base.cc:1034] running on GCE node
I20260810 00:56:26.654558 8227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:26.655369 8227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:26.656575 8227 hybrid_clock.cc:648] HybridClock initialized: now 1786323386656520 us; error 45 us; skew 500 ppm
I20260810 00:56:26.656857 8227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:26 dist-test-slave-jlzn krb5kdc[7646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323386, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:26.659698 8227 init.cc:377] Logged in from keytab as kudu/127.10.97.67@KRBTEST.COM (short username kudu)
I20260810 00:56:26.660962 8227 webserver.cc:460] Webserver started at http://127.10.97.67:34873/ using document root <none> and password file <none>
I20260810 00:56:26.661535 8227 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:26.661633 8227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:26.661878 8227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:26.664506 8227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "0555ed51b8c34937b28f2a41fa022b60"
format_stamp: "Formatted at 2026-08-10 00:56:26 on dist-test-slave-jlzn"
server_key: "3bbe4a521afc3c7707930a6dc6bbb51d"
server_key_iv: "eec1caf36105579dd9281d21c3e7f330"
server_key_version: "encryptionkey@0"
I20260810 00:56:26.665176 8227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "0555ed51b8c34937b28f2a41fa022b60"
format_stamp: "Formatted at 2026-08-10 00:56:26 on dist-test-slave-jlzn"
server_key: "3bbe4a521afc3c7707930a6dc6bbb51d"
server_key_iv: "eec1caf36105579dd9281d21c3e7f330"
server_key_version: "encryptionkey@0"
I20260810 00:56:26.669844 8227 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:56:26.672950 8250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:26.674103 8227 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:56:26.674223 8227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "0555ed51b8c34937b28f2a41fa022b60"
format_stamp: "Formatted at 2026-08-10 00:56:26 on dist-test-slave-jlzn"
server_key: "3bbe4a521afc3c7707930a6dc6bbb51d"
server_key_iv: "eec1caf36105579dd9281d21c3e7f330"
server_key_version: "encryptionkey@0"
I20260810 00:56:26.674381 8227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:26.694327 8227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:26.697726 8227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:26.697942 8227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:26.698715 8227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:56:26.699872 8227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:56:26.699949 8227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:26.700033 8227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:56:26.700075 8227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:26.729368 8227 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:38411
I20260810 00:56:26.729427 8433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:38411 every 8 connection(s)
I20260810 00:56:26.730455 8227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:56:26.733131 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 8227
I20260810 00:56:26.733332 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:56:26.733742 10629 external_mini_cluster.cc:1428] Setting key 1194607830d6165d2db92047ec919f37
Aug 10 00:56:26 dist-test-slave-jlzn krb5kdc[7646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323386, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.67@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:56:26.748628 8013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:26.732723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:45461 (local address 127.10.97.126:41297)
0810 00:56:26.732917 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:26.732924 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:26.734123 (+ 1199us) server_negotiation.cc:407] Connection header received
0810 00:56:26.735998 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:26.736003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:26.736085 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:26.736191 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:26.737528 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.738267 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.739702 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.740147 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.740238 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:26.743196 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:26.743201 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:26.743204 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:26.743234 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:26.745657 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:26.746290 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:26.746293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:26.746296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:26.746412 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:26.746832 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:26.746837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:26.746840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:26.747195 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:26.747335 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:26.748101 (+ 766us) server_negotiation.cc:299] Negotiation successful
0810 00:56:26.748319 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 00:56:26.749408 8436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:26.733179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:41297 (local address 127.10.97.67:45461)
0810 00:56:26.733965 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:26.734008 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:26.735733 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:26.736420 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:26.736431 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:26.736900 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:56:26.737399 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:26.737415 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.738369 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.738372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:26.739598 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:26.739610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.740331 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.740339 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:26.740561 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:26.741049 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:56:26.741074 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:56:26.743017 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:56:26.745811 (+ 2794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:56:26.745817 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:56:26.745826 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:56:26.746197 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:56:26.746490 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:56:26.746494 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:56:26.746498 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:56:26.746736 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:56:26.747322 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:56:26.747329 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:56:26.747703 (+ 374us) client_negotiation.cc:769] Sending connection context
0810 00:56:26.748012 (+ 309us) client_negotiation.cc:240] Negotiation successful
0810 00:56:26.748414 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":313,"threads_started":1}
I20260810 00:56:26.750531 8434 heartbeater.cc:347] Connected to a master server at 127.10.97.126:41297
I20260810 00:56:26.750783 8434 heartbeater.cc:464] Registering TS with master...
I20260810 00:56:26.751417 8434 heartbeater.cc:511] Master 127.10.97.126:41297 requested a full tablet report, sending...
I20260810 00:56:26.752770 7714 ts_manager.cc:194] Registered new tserver with Master: 0555ed51b8c34937b28f2a41fa022b60 (127.10.97.67:38411)
I20260810 00:56:26.753526 7714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.97.67@KRBTEST.COM'} at 127.10.97.67:45461
I20260810 00:56:26.760381 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:56:26.775107 8013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:26.764309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42424 (local address 127.10.97.126:41297)
0810 00:56:26.764444 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:26.764448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:26.764896 (+ 448us) server_negotiation.cc:407] Connection header received
0810 00:56:26.765020 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:26.765023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:26.765086 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:26.765163 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:26.766441 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.767106 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.768287 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.768680 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.768769 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:26.770797 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:26.770801 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:26.770804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:26.770826 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:26.773107 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:26.773630 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:26.773641 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:26.773643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:26.773761 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:26.774074 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:26.774077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:26.774079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:26.774338 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:26.774509 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:26.774751 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 00:56:26.774896 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 00:56:26.778524 7714 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42424:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:56:26.784904 7714 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:56:26.786147 8010 heartbeater.cc:503] Master 127.10.97.126:41297 was elected leader, sending a full tablet report...
I20260810 00:56:26.812292 8452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:26.795780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:39021 (local address 127.0.0.1:37086)
0810 00:56:26.796661 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:26.796695 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:26.796871 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:26.804493 (+ 7622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:26.804500 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:26.804530 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:26.804909 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:26.804922 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.807127 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.807144 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:26.809611 (+ 2467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:26.809629 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.811415 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.811422 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:26.811640 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:26.811686 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 00:56:26.811772 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 00:56:26.811948 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":458,"threads_started":1}
I20260810 00:56:26.812678 8454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:26.796054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37086 (local address 127.10.97.65:39021)
0810 00:56:26.802952 (+ 6898us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:26.802958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:26.802985 (+ 27us) server_negotiation.cc:407] Connection header received
0810 00:56:26.803068 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:26.803075 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:26.803324 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:26.805153 (+ 1829us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:26.805377 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.806954 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.809829 (+ 2875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.811194 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.811927 (+ 733us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:26.812153 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:26.812221 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 00:56:26.812372 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6770,"thread_start_us":6615,"threads_started":1}
I20260810 00:56:26.816418 7906 tablet_service.cc:1467] Processing CreateTablet for tablet 281970ac792a45069e51503d0765cfec (DEFAULT_TABLE table=test-table [id=2e24b8f1af47406fb3c085e289076e14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:56:26.817157 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:26.796046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:46735 (local address 127.0.0.1:56450)
0810 00:56:26.809162 (+ 13116us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:26.809184 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:26.809300 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:26.810739 (+ 1439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:26.810743 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:26.810768 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:26.811223 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:26.811237 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.813193 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.813198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:26.814730 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:26.814742 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.816448 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.816452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:26.816557 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:26.816579 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 00:56:26.816655 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 00:56:26.816769 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12947,"mutex_wait_us":3,"thread_start_us":13330,"threads_started":1}
I20260810 00:56:26.817158 8457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:26.796574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56450 (local address 127.10.97.66:46735)
0810 00:56:26.809980 (+ 13406us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:26.809989 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:26.810018 (+ 29us) server_negotiation.cc:407] Connection header received
0810 00:56:26.810117 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:26.810125 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:26.810330 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:26.810508 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:26.811403 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.813053 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.814880 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.816060 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.816182 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:26.816364 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:26.816719 (+ 355us) server_negotiation.cc:299] Negotiation successful
0810 00:56:26.816869 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13231,"thread_start_us":13092,"threads_started":1}
I20260810 00:56:26.818128 7906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 281970ac792a45069e51503d0765cfec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:26.818269 8458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:26.797869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:38411 (local address 127.0.0.1:50418)
0810 00:56:26.810235 (+ 12366us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:26.810253 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:26.810374 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:26.811312 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:26.811316 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:26.811342 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:26.811775 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:26.811787 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.813549 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.813555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:26.815637 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:26.815652 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.817514 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.817520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:26.817695 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:26.817726 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 00:56:26.817819 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 00:56:26.817933 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12253,"mutex_wait_us":1,"thread_start_us":12123,"threads_started":1}
I20260810 00:56:26.818356 8459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:26.798435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50418 (local address 127.10.97.67:38411)
0810 00:56:26.807786 (+ 9351us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:26.807795 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:26.810357 (+ 2562us) server_negotiation.cc:407] Connection header received
0810 00:56:26.810682 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:26.810689 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:26.810882 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:26.811057 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:26.812013 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.813423 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.815848 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:26.817181 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:26.817299 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:26.817505 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:26.817878 (+ 373us) server_negotiation.cc:299] Negotiation successful
0810 00:56:26.818028 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9185,"thread_start_us":9036,"threads_started":1}
I20260810 00:56:26.820142 8109 tablet_service.cc:1467] Processing CreateTablet for tablet 281970ac792a45069e51503d0765cfec (DEFAULT_TABLE table=test-table [id=2e24b8f1af47406fb3c085e289076e14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:56:26.821244 8109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 281970ac792a45069e51503d0765cfec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:26.823881 8328 tablet_service.cc:1467] Processing CreateTablet for tablet 281970ac792a45069e51503d0765cfec (DEFAULT_TABLE table=test-table [id=2e24b8f1af47406fb3c085e289076e14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:56:26.825203 8328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 281970ac792a45069e51503d0765cfec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:26.831758 8460 tablet_bootstrap.cc:492] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966: Bootstrap starting.
I20260810 00:56:26.833518 8461 tablet_bootstrap.cc:492] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9: Bootstrap starting.
I20260810 00:56:26.835235 8460 tablet_bootstrap.cc:654] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:26.836509 8460 log.cc:824] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:26.836865 8461 tablet_bootstrap.cc:654] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:26.838205 8461 log.cc:824] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:26.839046 8463 tablet_bootstrap.cc:492] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60: Bootstrap starting.
I20260810 00:56:26.842021 8463 tablet_bootstrap.cc:654] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:26.842239 8460 tablet_bootstrap.cc:492] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966: No bootstrap required, opened a new log
I20260810 00:56:26.842238 8461 tablet_bootstrap.cc:492] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9: No bootstrap required, opened a new log
I20260810 00:56:26.842506 8460 ts_tablet_manager.cc:1397] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260810 00:56:26.842501 8461 ts_tablet_manager.cc:1397] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 00:56:26.842955 8463 log.cc:824] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:26.845170 8463 tablet_bootstrap.cc:492] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60: No bootstrap required, opened a new log
I20260810 00:56:26.845391 8463 ts_tablet_manager.cc:1397] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 00:56:26.852586 8461 raft_consensus.cc:348] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93837d6279904ce29294a7d61e5406a9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39021 } } peers { permanent_uuid: "6636df8556f64f3982d53425f36ce966" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46735 } } peers { permanent_uuid: "0555ed51b8c34937b28f2a41fa022b60" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38411 } }
I20260810 00:56:26.852586 8460 raft_consensus.cc:348] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93837d6279904ce29294a7d61e5406a9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39021 } } peers { permanent_uuid: "6636df8556f64f3982d53425f36ce966" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46735 } } peers { permanent_uuid: "0555ed51b8c34937b28f2a41fa022b60" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38411 } }
I20260810 00:56:26.853003 8460 raft_consensus.cc:374] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:26.853003 8461 raft_consensus.cc:374] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:26.853101 8461 raft_consensus.cc:729] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93837d6279904ce29294a7d61e5406a9, State: Initialized, Role: FOLLOWER
I20260810 00:56:26.852883 8463 raft_consensus.cc:348] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93837d6279904ce29294a7d61e5406a9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39021 } } peers { permanent_uuid: "6636df8556f64f3982d53425f36ce966" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46735 } } peers { permanent_uuid: "0555ed51b8c34937b28f2a41fa022b60" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38411 } }
I20260810 00:56:26.853152 8460 raft_consensus.cc:729] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6636df8556f64f3982d53425f36ce966, State: Initialized, Role: FOLLOWER
I20260810 00:56:26.853245 8463 raft_consensus.cc:374] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:26.853356 8463 raft_consensus.cc:729] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0555ed51b8c34937b28f2a41fa022b60, State: Initialized, Role: FOLLOWER
I20260810 00:56:26.853709 8461 consensus_queue.cc:260] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93837d6279904ce29294a7d61e5406a9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39021 } } peers { permanent_uuid: "6636df8556f64f3982d53425f36ce966" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46735 } } peers { permanent_uuid: "0555ed51b8c34937b28f2a41fa022b60" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38411 } }
I20260810 00:56:26.853713 8460 consensus_queue.cc:260] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93837d6279904ce29294a7d61e5406a9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39021 } } peers { permanent_uuid: "6636df8556f64f3982d53425f36ce966" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46735 } } peers { permanent_uuid: "0555ed51b8c34937b28f2a41fa022b60" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38411 } }
I20260810 00:56:26.853966 8463 consensus_queue.cc:260] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93837d6279904ce29294a7d61e5406a9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39021 } } peers { permanent_uuid: "6636df8556f64f3982d53425f36ce966" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46735 } } peers { permanent_uuid: "0555ed51b8c34937b28f2a41fa022b60" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38411 } }
I20260810 00:56:26.855829 8460 ts_tablet_manager.cc:1428] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260810 00:56:26.856081 8220 heartbeater.cc:503] Master 127.10.97.126:41297 was elected leader, sending a full tablet report...
I20260810 00:56:26.856894 8461 ts_tablet_manager.cc:1428] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260810 00:56:26.857800 8434 heartbeater.cc:503] Master 127.10.97.126:41297 was elected leader, sending a full tablet report...
I20260810 00:56:26.858352 8463 ts_tablet_manager.cc:1428] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
W20260810 00:56:26.981877 8435 tablet.cc:2369] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:56:26.996044 8221 tablet.cc:2369] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:56:27.004704 8468 raft_consensus.cc:482] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:56:27.004936 8468 raft_consensus.cc:504] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93837d6279904ce29294a7d61e5406a9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39021 } } peers { permanent_uuid: "6636df8556f64f3982d53425f36ce966" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46735 } } peers { permanent_uuid: "0555ed51b8c34937b28f2a41fa022b60" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38411 } }
I20260810 00:56:27.006500 8468 leader_election.cc:288] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93837d6279904ce29294a7d61e5406a9 (127.10.97.65:39021), 0555ed51b8c34937b28f2a41fa022b60 (127.10.97.67:38411)
W20260810 00:56:27.008656 8012 tablet.cc:2369] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:56:27.014425 8454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:27.006950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:54699 (local address 127.10.97.65:39021)
0810 00:56:27.007122 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:27.007127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:27.007890 (+ 763us) server_negotiation.cc:407] Connection header received
0810 00:56:27.008077 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:27.008082 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:27.008160 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:27.008252 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:27.008950 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:27.009752 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:27.012645 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:27.013462 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:27.013580 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:27.013662 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:27.014061 (+ 399us) server_negotiation.cc:299] Negotiation successful
0810 00:56:27.014188 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 00:56:27.014441 8459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:27.006879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:38785 (local address 127.10.97.67:38411)
0810 00:56:27.007065 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:27.007071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:27.007890 (+ 819us) server_negotiation.cc:407] Connection header received
0810 00:56:27.008082 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:27.008086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:27.008166 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:27.008252 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:27.008988 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:27.009871 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:27.012678 (+ 2807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:27.013518 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:27.013685 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:27.013756 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:27.014089 (+ 333us) server_negotiation.cc:299] Negotiation successful
0810 00:56:27.014210 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 00:56:27.014652 8474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:27.006795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:39021 (local address 127.10.97.66:54699)
0810 00:56:27.007744 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:27.007767 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:27.007938 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:27.008285 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:27.008290 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:27.008319 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:27.008755 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:27.008772 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:27.009895 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:27.009898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:27.012477 (+ 2579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:27.012492 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:27.013685 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:27.013692 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:27.013895 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:27.013929 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 00:56:27.014002 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 00:56:27.014125 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":804,"mutex_wait_us":130,"thread_start_us":485,"threads_started":1}
I20260810 00:56:27.014726 8475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:27.006832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:38411 (local address 127.10.97.66:38785)
0810 00:56:27.007771 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:27.007784 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:27.007968 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:27.008285 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:27.008291 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:27.008332 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:27.008836 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:27.008847 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:27.009999 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:27.010003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:27.012478 (+ 2475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:27.012492 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:27.013743 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:27.013748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:27.013933 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:27.013959 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 00:56:27.014050 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 00:56:27.014184 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"mutex_wait_us":124,"spinlock_wait_cycles":47872,"thread_start_us":448,"threads_started":1}
I20260810 00:56:27.015615 8363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "281970ac792a45069e51503d0765cfec" candidate_uuid: "6636df8556f64f3982d53425f36ce966" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0555ed51b8c34937b28f2a41fa022b60" is_pre_election: true
I20260810 00:56:27.015640 7936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "281970ac792a45069e51503d0765cfec" candidate_uuid: "6636df8556f64f3982d53425f36ce966" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93837d6279904ce29294a7d61e5406a9" is_pre_election: true
I20260810 00:56:27.016093 7936 raft_consensus.cc:2454] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6636df8556f64f3982d53425f36ce966 in term 0.
I20260810 00:56:27.016106 8363 raft_consensus.cc:2454] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6636df8556f64f3982d53425f36ce966 in term 0.
I20260810 00:56:27.016691 8041 leader_election.cc:302] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0555ed51b8c34937b28f2a41fa022b60, 6636df8556f64f3982d53425f36ce966; no voters:
I20260810 00:56:27.016995 8468 raft_consensus.cc:2781] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:56:27.017127 8468 raft_consensus.cc:482] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:56:27.017211 8468 raft_consensus.cc:3037] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:27.019266 8468 raft_consensus.cc:504] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93837d6279904ce29294a7d61e5406a9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39021 } } peers { permanent_uuid: "6636df8556f64f3982d53425f36ce966" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46735 } } peers { permanent_uuid: "0555ed51b8c34937b28f2a41fa022b60" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38411 } }
I20260810 00:56:27.019776 8468 leader_election.cc:288] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [CANDIDATE]: Term 1 election: Requested vote from peers 93837d6279904ce29294a7d61e5406a9 (127.10.97.65:39021), 0555ed51b8c34937b28f2a41fa022b60 (127.10.97.67:38411)
I20260810 00:56:27.020162 7936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "281970ac792a45069e51503d0765cfec" candidate_uuid: "6636df8556f64f3982d53425f36ce966" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93837d6279904ce29294a7d61e5406a9"
I20260810 00:56:27.020198 8363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "281970ac792a45069e51503d0765cfec" candidate_uuid: "6636df8556f64f3982d53425f36ce966" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0555ed51b8c34937b28f2a41fa022b60"
I20260810 00:56:27.020316 7936 raft_consensus.cc:3037] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:27.020364 8363 raft_consensus.cc:3037] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:27.022351 7936 raft_consensus.cc:2454] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6636df8556f64f3982d53425f36ce966 in term 1.
I20260810 00:56:27.022469 8363 raft_consensus.cc:2454] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6636df8556f64f3982d53425f36ce966 in term 1.
I20260810 00:56:27.022716 8040 leader_election.cc:302] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6636df8556f64f3982d53425f36ce966, 93837d6279904ce29294a7d61e5406a9; no voters:
I20260810 00:56:27.023015 8468 raft_consensus.cc:2781] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:27.023433 8468 raft_consensus.cc:686] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [term 1 LEADER]: Becoming Leader. State: Replica: 6636df8556f64f3982d53425f36ce966, State: Running, Role: LEADER
I20260810 00:56:27.023881 8468 consensus_queue.cc:237] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93837d6279904ce29294a7d61e5406a9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39021 } } peers { permanent_uuid: "6636df8556f64f3982d53425f36ce966" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46735 } } peers { permanent_uuid: "0555ed51b8c34937b28f2a41fa022b60" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38411 } }
I20260810 00:56:27.028152 7714 catalog_manager.cc:5495] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6636df8556f64f3982d53425f36ce966 (127.10.97.66). New cstate: current_term: 1 leader_uuid: "6636df8556f64f3982d53425f36ce966" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93837d6279904ce29294a7d61e5406a9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6636df8556f64f3982d53425f36ce966" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0555ed51b8c34937b28f2a41fa022b60" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38411 } health_report { overall_health: UNKNOWN } } }
I20260810 00:56:27.097500 8457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:27.092764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56462 (local address 127.10.97.66:46735)
0810 00:56:27.092925 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:27.092930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:27.093020 (+ 90us) server_negotiation.cc:407] Connection header received
0810 00:56:27.093194 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:27.093198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:27.093270 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:27.093385 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:56:27.093660 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:27.094351 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:27.095580 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:27.096004 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:27.096092 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:27.096527 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:56:27.097045 (+ 518us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:56:27.097152 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:27.097221 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 00:56:27.097281 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 00:56:27.104830 7936 raft_consensus.cc:1261] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9 [term 1 FOLLOWER]: Refusing update from remote peer 6636df8556f64f3982d53425f36ce966: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:56:27.104830 8363 raft_consensus.cc:1261] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60 [term 1 FOLLOWER]: Refusing update from remote peer 6636df8556f64f3982d53425f36ce966: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:56:27.105708 8476 consensus_queue.cc:1035] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93837d6279904ce29294a7d61e5406a9" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:56:27.106014 8468 consensus_queue.cc:1035] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0555ed51b8c34937b28f2a41fa022b60" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 38411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:56:27.116663 8482 mvcc.cc:205] Tried to move back new op lower bound from 7316780593572491264 to 7316780593253015552. Current Snapshot: MvccSnapshot[applied={T|T < 7316780593572491264}]
I20260810 00:56:27.130219 7714 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:56:27.130430 7714 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:56:27.133623 7714 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac1101e047bb4e58bca916302fab44df: Sending DeleteTablet for 3 replicas of tablet 281970ac792a45069e51503d0765cfec
I20260810 00:56:27.134651 7906 tablet_service.cc:1514] Processing DeleteTablet for tablet 281970ac792a45069e51503d0765cfec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:56:27 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:37086
I20260810 00:56:27.134716 8109 tablet_service.cc:1514] Processing DeleteTablet for tablet 281970ac792a45069e51503d0765cfec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:56:27 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:56450
I20260810 00:56:27.135366 8328 tablet_service.cc:1514] Processing DeleteTablet for tablet 281970ac792a45069e51503d0765cfec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:56:27 UTC) from {username='kudu', principal='kudu/127.10.97.126@KRBTEST.COM'} at 127.0.0.1:50418
I20260810 00:56:27.137180 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 7815
I20260810 00:56:27.148265 8496 tablet_replica.cc:330] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60: stopping tablet replica
I20260810 00:56:27.148681 8496 raft_consensus.cc:2229] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:56:27.149174 8496 raft_consensus.cc:2258] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:56:27.150082 8494 tablet_replica.cc:330] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9: stopping tablet replica
I20260810 00:56:27.150455 8494 raft_consensus.cc:2229] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:56:27.150904 8494 raft_consensus.cc:2258] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:56:27.152113 8496 ts_tablet_manager.cc:1905] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:56:27.152957 8494 ts_tablet_manager.cc:1905] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:56:27.158898 8496 ts_tablet_manager.cc:1918] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:56:27.159094 8496 log.cc:1196] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/281970ac792a45069e51503d0765cfec
Aug 10 00:56:27 dist-test-slave-jlzn krb5kdc[7646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.65@KRBTEST.COM
I20260810 00:56:27.159696 8496 ts_tablet_manager.cc:1939] T 281970ac792a45069e51503d0765cfec P 0555ed51b8c34937b28f2a41fa022b60: Deleting consensus metadata
I20260810 00:56:27.161466 7692 catalog_manager.cc:4841] TS 0555ed51b8c34937b28f2a41fa022b60 (127.10.97.67:38411): tablet 281970ac792a45069e51503d0765cfec (table test-table [id=2e24b8f1af47406fb3c085e289076e14]) successfully deleted
I20260810 00:56:27.162484 8495 tablet_replica.cc:330] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966: stopping tablet replica
I20260810 00:56:27.162844 8495 raft_consensus.cc:2229] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:56:27.163568 8495 raft_consensus.cc:2258] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:56:27.164412 8494 ts_tablet_manager.cc:1918] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:56:27.164631 8494 log.cc:1196] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-0/wal/wals/281970ac792a45069e51503d0765cfec
I20260810 00:56:27.165185 8494 ts_tablet_manager.cc:1939] T 281970ac792a45069e51503d0765cfec P 93837d6279904ce29294a7d61e5406a9: Deleting consensus metadata
I20260810 00:56:27.165642 8495 ts_tablet_manager.cc:1905] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:56:27.166951 7691 catalog_manager.cc:4841] TS 93837d6279904ce29294a7d61e5406a9 (127.10.97.65:39021): tablet 281970ac792a45069e51503d0765cfec (table test-table [id=2e24b8f1af47406fb3c085e289076e14]) successfully deleted
I20260810 00:56:27.167847 8454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:27.138232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37090 (local address 127.10.97.65:39021)
0810 00:56:27.138407 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:27.138413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:27.149375 (+ 10962us) server_negotiation.cc:407] Connection header received
0810 00:56:27.149714 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:27.149720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:27.149804 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:27.149934 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:27.151537 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:27.152679 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:27.154423 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:27.155057 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:27.155149 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:27.160858 (+ 5709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:27.160875 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:27.160892 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:27.160932 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:27.164427 (+ 3495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:27.165266 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:27.165274 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:27.165278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:27.165470 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:27.165941 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:27.165946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:27.165949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:27.166404 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:27.166683 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:27.167252 (+ 569us) server_negotiation.cc:299] Negotiation successful
0810 00:56:27.167454 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 00:56:27.169216 8002 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:27.173422 8495 ts_tablet_manager.cc:1918] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:56:27.173568 8495 log.cc:1196] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/281970ac792a45069e51503d0765cfec
I20260810 00:56:27.173990 8495 ts_tablet_manager.cc:1939] T 281970ac792a45069e51503d0765cfec P 6636df8556f64f3982d53425f36ce966: Deleting consensus metadata
I20260810 00:56:27.175204 7694 catalog_manager.cc:4841] TS 6636df8556f64f3982d53425f36ce966 (127.10.97.66:46735): tablet 281970ac792a45069e51503d0765cfec (table test-table [id=2e24b8f1af47406fb3c085e289076e14]) successfully deleted
I20260810 00:56:27.284453 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 7815
I20260810 00:56:27.307418 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 8018
Aug 10 00:56:27 dist-test-slave-jlzn krb5kdc[7646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.66@KRBTEST.COM
I20260810 00:56:27.323024 8457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:27.308251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56468 (local address 127.10.97.66:46735)
0810 00:56:27.308437 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:27.308443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:27.308463 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:56:27.308619 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:27.308624 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:27.308708 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:27.308793 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:27.310062 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:27.310887 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:27.312268 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:27.312805 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:27.312927 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:27.317242 (+ 4315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:27.317258 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:27.317272 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:27.317312 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:27.320328 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:27.321096 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:27.321103 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:27.321105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:27.321238 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:27.321588 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:27.321591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:27.321593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:27.321932 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:27.322183 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:27.322452 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 00:56:27.322617 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 00:56:27.323724 8212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:27.431885 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 8018
I20260810 00:56:27.456115 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 8227
Aug 10 00:56:27 dist-test-slave-jlzn krb5kdc[7646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.67@KRBTEST.COM
I20260810 00:56:27.470502 8459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:27.456841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50430 (local address 127.10.97.67:38411)
0810 00:56:27.457019 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:27.457025 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:27.457044 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:56:27.457233 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:27.457237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:27.457303 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:27.457399 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:27.458722 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:27.459437 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:27.460713 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:27.461099 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:27.461208 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:27.465422 (+ 4214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:27.465436 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:27.465452 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:27.465484 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:27.467916 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:27.468580 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:27.468588 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:27.468590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:27.468731 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:27.469076 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:27.469080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:27.469082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:27.469424 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:27.469719 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:27.469998 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 00:56:27.470202 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 00:56:27.471125 8429 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:27.557005 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 8227
I20260810 00:56:27.576968 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 7665
I20260810 00:56:27.577999 7755 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:27.683606 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 7665
2026-08-10T00:56:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4357 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T00:56:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:27Z Disabled control of system clock
I20260810 00:56:27.767756 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:34487
--webserver_interface=127.10.97.126
--webserver_port=0
--builtin_ntp_servers=127.10.97.84:45145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:34487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 00:56:28.090943 8509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:28.091202 8509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:28.091321 8509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:28.098243 8509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:28.098337 8509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:28.098402 8509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:28.098475 8509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:28.098532 8509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:28.107197 8509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:45145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:34487
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:34487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:28.108304 8509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:28.109920 8509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:28.120862 8515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:28.120819 8519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:28.120890 8517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:28.121587 8509 server_base.cc:1034] running on GCE node
I20260810 00:56:28.122316 8509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:28.123538 8509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:28.124724 8509 hybrid_clock.cc:648] HybridClock initialized: now 1786323388124677 us; error 37 us; skew 500 ppm
I20260810 00:56:28.125034 8509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:56:28.127085 8509 webserver.cc:460] Webserver started at http://127.10.97.126:34997/ using document root <none> and password file <none>
I20260810 00:56:28.127683 8509 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:28.127775 8509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:28.128093 8509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:28.130779 8509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "c7bff0cc45114bd0bded7c5ef7f7d6ed"
format_stamp: "Formatted at 2026-08-10 00:56:28 on dist-test-slave-jlzn"
server_key: "7ef1d75555e4175cd8887b1b83e40536"
server_key_iv: "2ee0e98523bed28b272d011fe370f0b2"
server_key_version: "encryptionkey@0"
I20260810 00:56:28.131544 8509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "c7bff0cc45114bd0bded7c5ef7f7d6ed"
format_stamp: "Formatted at 2026-08-10 00:56:28 on dist-test-slave-jlzn"
server_key: "7ef1d75555e4175cd8887b1b83e40536"
server_key_iv: "2ee0e98523bed28b272d011fe370f0b2"
server_key_version: "encryptionkey@0"
I20260810 00:56:28.136193 8509 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260810 00:56:28.139314 8525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:28.140540 8509 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:56:28.140704 8509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "c7bff0cc45114bd0bded7c5ef7f7d6ed"
format_stamp: "Formatted at 2026-08-10 00:56:28 on dist-test-slave-jlzn"
server_key: "7ef1d75555e4175cd8887b1b83e40536"
server_key_iv: "2ee0e98523bed28b272d011fe370f0b2"
server_key_version: "encryptionkey@0"
I20260810 00:56:28.140897 8509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:28.153679 8509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:28.154379 8509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:28.154632 8509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:28.170971 8509 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:34487
I20260810 00:56:28.171002 8603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:34487 every 8 connection(s)
I20260810 00:56:28.172202 8509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:28.175653 8604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:28.177702 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 8509
I20260810 00:56:28.177847 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:28.178290 10629 external_mini_cluster.cc:1428] Setting key 54dbfd7f7fce3d76f2a25131a9ce2f1c
I20260810 00:56:28.186578 8604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed: Bootstrap starting.
I20260810 00:56:28.189873 8604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:28.190927 8604 log.cc:824] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:28.191201 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:28.180084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52994 (local address 127.10.97.126:34487)
0810 00:56:28.181052 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:28.181062 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:28.181101 (+ 39us) server_negotiation.cc:407] Connection header received
0810 00:56:28.182700 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:28.182721 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:28.183124 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:28.183436 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:28.184061 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:28.185324 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:28.187131 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:28.187766 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:28.187940 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:28.189090 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:28.189097 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:28.189109 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:28.189592 (+ 483us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:28.189675 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:28.189700 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:28.189784 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:28.190040 (+ 256us) server_negotiation.cc:299] Negotiation successful
0810 00:56:28.190341 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":352,"threads_started":1}
I20260810 00:56:28.193454 8604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed: No bootstrap required, opened a new log
I20260810 00:56:28.200109 8604 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bff0cc45114bd0bded7c5ef7f7d6ed" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 34487 } }
I20260810 00:56:28.200424 8604 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:28.200510 8604 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bff0cc45114bd0bded7c5ef7f7d6ed, State: Initialized, Role: FOLLOWER
I20260810 00:56:28.201027 8604 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bff0cc45114bd0bded7c5ef7f7d6ed" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 34487 } }
I20260810 00:56:28.201258 8604 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:28.201385 8604 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:28.201541 8604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:28.203485 8604 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bff0cc45114bd0bded7c5ef7f7d6ed" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 34487 } }
I20260810 00:56:28.203994 8604 leader_election.cc:302] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7bff0cc45114bd0bded7c5ef7f7d6ed; no voters:
I20260810 00:56:28.204502 8604 leader_election.cc:288] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:28.204619 8610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:28.205295 8610 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed [term 1 LEADER]: Becoming Leader. State: Replica: c7bff0cc45114bd0bded7c5ef7f7d6ed, State: Running, Role: LEADER
I20260810 00:56:28.205750 8610 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bff0cc45114bd0bded7c5ef7f7d6ed" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 34487 } }
I20260810 00:56:28.206255 8604 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:28.208380 8612 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7bff0cc45114bd0bded7c5ef7f7d6ed. Latest consensus state: current_term: 1 leader_uuid: "c7bff0cc45114bd0bded7c5ef7f7d6ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bff0cc45114bd0bded7c5ef7f7d6ed" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 34487 } } }
I20260810 00:56:28.208753 8612 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:28.209466 8611 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7bff0cc45114bd0bded7c5ef7f7d6ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bff0cc45114bd0bded7c5ef7f7d6ed" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 34487 } } }
I20260810 00:56:28.209786 8611 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:28.210227 8618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:28.214095 8618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:28.222303 8618 catalog_manager.cc:1347] Generated new cluster ID: 34291ce1a021452eae74fb3b4f72e16e
I20260810 00:56:28.222389 8618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:28.234692 8618 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:28.235844 8618 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:28.245196 8618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed: Generated new TSK 0
I20260810 00:56:28.245822 8618 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:56:28.249799 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.65:0
--local_ip_for_outbound_sockets=127.10.97.65
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:34487
--builtin_ntp_servers=127.10.97.84:45145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 00:56:28.568994 8636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:28.569248 8636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:28.569368 8636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:28.576233 8636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:28.576319 8636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:28.576462 8636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.65
I20260810 00:56:28.584934 8636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:45145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.97.65
--webserver_port=0
--tserver_master_addrs=127.10.97.126:34487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.65
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:28.586018 8636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:28.587622 8636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:28.599382 8644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:28.599395 8645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:28.599831 8647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:28.600111 8636 server_base.cc:1034] running on GCE node
I20260810 00:56:28.600680 8636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:28.601477 8636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:28.602682 8636 hybrid_clock.cc:648] HybridClock initialized: now 1786323388602638 us; error 33 us; skew 500 ppm
I20260810 00:56:28.602958 8636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:56:28.605086 8636 webserver.cc:460] Webserver started at http://127.10.97.65:41249/ using document root <none> and password file <none>
I20260810 00:56:28.605690 8636 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:28.605782 8636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:28.606067 8636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:28.608737 8636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-0/data/instance:
uuid: "4cc01db279414490a0f2487e9859f991"
format_stamp: "Formatted at 2026-08-10 00:56:28 on dist-test-slave-jlzn"
server_key: "facd0b819687be2e4ba5f7447e2c8623"
server_key_iv: "6839f6ec19217ae0df5702c578edc8b4"
server_key_version: "encryptionkey@0"
I20260810 00:56:28.609468 8636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance:
uuid: "4cc01db279414490a0f2487e9859f991"
format_stamp: "Formatted at 2026-08-10 00:56:28 on dist-test-slave-jlzn"
server_key: "facd0b819687be2e4ba5f7447e2c8623"
server_key_iv: "6839f6ec19217ae0df5702c578edc8b4"
server_key_version: "encryptionkey@0"
I20260810 00:56:28.614048 8636 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:56:28.617264 8654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:28.618404 8636 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:56:28.618546 8636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-0/wal
uuid: "4cc01db279414490a0f2487e9859f991"
format_stamp: "Formatted at 2026-08-10 00:56:28 on dist-test-slave-jlzn"
server_key: "facd0b819687be2e4ba5f7447e2c8623"
server_key_iv: "6839f6ec19217ae0df5702c578edc8b4"
server_key_version: "encryptionkey@0"
I20260810 00:56:28.618733 8636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:28.635408 8636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:28.636092 8636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:28.636363 8636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:28.637220 8636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:56:28.638489 8636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:56:28.638608 8636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:28.638713 8636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:56:28.638787 8636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:28.668079 8636 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.65:46121
I20260810 00:56:28.668138 8824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.65:46121 every 8 connection(s)
I20260810 00:56:28.669257 8636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-0/data/info.pb
I20260810 00:56:28.677407 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 8636
I20260810 00:56:28.677560 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-0/wal/instance
I20260810 00:56:28.678056 10629 external_mini_cluster.cc:1428] Setting key d0e721abbcad9404618fdd6e5406ac09
I20260810 00:56:28.680780 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:28.671545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:37097 (local address 127.10.97.126:34487)
0810 00:56:28.671710 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:28.671714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:28.672940 (+ 1226us) server_negotiation.cc:407] Connection header received
0810 00:56:28.674574 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:28.674578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:28.674649 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:28.674730 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:28.675656 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:28.676402 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:28.677926 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:28.678529 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:28.678659 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:28.680026 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:28.680032 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:28.680036 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:28.680056 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:28.680093 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:28.680108 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:28.680207 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:28.680383 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 00:56:28.680475 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 00:56:28.681015 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.66:0
--local_ip_for_outbound_sockets=127.10.97.66
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:34487
--builtin_ntp_servers=127.10.97.84:45145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 00:56:28.681593 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:28.671959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:34487 (local address 127.10.97.65:37097)
0810 00:56:28.672796 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:28.672833 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:28.674357 (+ 1524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:28.674906 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:28.674913 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:28.675001 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:56:28.675523 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:28.675540 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:28.676522 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:28.676526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:28.677785 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:28.677797 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:28.678760 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:28.678768 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:28.679003 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:28.679666 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:56:28.679669 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:56:28.679725 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:56:28.679849 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:56:28.680206 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:56:28.680215 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:56:28.680220 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 00:56:28.680350 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 00:56:28.680616 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":252,"threads_started":1}
I20260810 00:56:28.682981 8826 heartbeater.cc:347] Connected to a master server at 127.10.97.126:34487
I20260810 00:56:28.683269 8826 heartbeater.cc:464] Registering TS with master...
I20260810 00:56:28.684082 8826 heartbeater.cc:511] Master 127.10.97.126:34487 requested a full tablet report, sending...
I20260810 00:56:28.685982 8546 ts_manager.cc:194] Registered new tserver with Master: 4cc01db279414490a0f2487e9859f991 (127.10.97.65:46121)
I20260810 00:56:28.687435 8546 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.97.65:37097
W20260810 00:56:29.001210 8831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:29.001441 8831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:29.001567 8831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:29.008558 8831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:29.008644 8831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:29.008778 8831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.66
I20260810 00:56:29.017350 8831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:45145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.97.66
--webserver_port=0
--tserver_master_addrs=127.10.97.126:34487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.66
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:29.018405 8831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:29.020015 8831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:29.032109 8837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:29.032156 8840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:29.032114 8836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:29.032989 8831 server_base.cc:1034] running on GCE node
I20260810 00:56:29.033484 8831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:29.034272 8831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:29.035509 8831 hybrid_clock.cc:648] HybridClock initialized: now 1786323389035439 us; error 38 us; skew 500 ppm
I20260810 00:56:29.035820 8831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:56:29.037863 8831 webserver.cc:460] Webserver started at http://127.10.97.66:42359/ using document root <none> and password file <none>
I20260810 00:56:29.038472 8831 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:29.038581 8831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:29.038872 8831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:29.041513 8831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-1/data/instance:
uuid: "ce2b06d499d9423185100b7929d11eca"
format_stamp: "Formatted at 2026-08-10 00:56:29 on dist-test-slave-jlzn"
server_key: "8eea280c24eb3bfd91f32cad0771d244"
server_key_iv: "0c8603353178be35887c75a206ea3a1f"
server_key_version: "encryptionkey@0"
I20260810 00:56:29.042241 8831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance:
uuid: "ce2b06d499d9423185100b7929d11eca"
format_stamp: "Formatted at 2026-08-10 00:56:29 on dist-test-slave-jlzn"
server_key: "8eea280c24eb3bfd91f32cad0771d244"
server_key_iv: "0c8603353178be35887c75a206ea3a1f"
server_key_version: "encryptionkey@0"
I20260810 00:56:29.046868 8831 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260810 00:56:29.050014 8853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:29.051167 8831 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:56:29.051326 8831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-1/wal
uuid: "ce2b06d499d9423185100b7929d11eca"
format_stamp: "Formatted at 2026-08-10 00:56:29 on dist-test-slave-jlzn"
server_key: "8eea280c24eb3bfd91f32cad0771d244"
server_key_iv: "0c8603353178be35887c75a206ea3a1f"
server_key_version: "encryptionkey@0"
I20260810 00:56:29.051542 8831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:29.065845 8831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:29.066546 8831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:29.066815 8831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:29.067641 8831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:56:29.068958 8831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:56:29.069056 8831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:29.069162 8831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:56:29.069233 8831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:29.100077 8831 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.66:46867
I20260810 00:56:29.100142 9030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.66:46867 every 8 connection(s)
I20260810 00:56:29.101215 8831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-1/data/info.pb
I20260810 00:56:29.108651 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 8831
I20260810 00:56:29.108804 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-1/wal/instance
I20260810 00:56:29.109361 10629 external_mini_cluster.cc:1428] Setting key a4c002260ec111d7bbd906872d5bf86e
I20260810 00:56:29.112190 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.97.67:0
--local_ip_for_outbound_sockets=127.10.97.67
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:34487
--builtin_ntp_servers=127.10.97.84:45145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 00:56:29.113304 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.103492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.66:49901 (local address 127.10.97.126:34487)
0810 00:56:29.103663 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:29.103667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:29.105014 (+ 1347us) server_negotiation.cc:407] Connection header received
0810 00:56:29.106828 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:29.106832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:29.106898 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:29.106980 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:29.107948 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.108687 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.110663 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.111104 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.111226 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:29.112537 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:29.112542 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:29.112545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:29.112561 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:29.112594 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:29.112605 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:29.112686 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:29.112897 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 00:56:29.112991 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 00:56:29.114171 9036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.104043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:34487 (local address 127.10.97.66:49901)
0810 00:56:29.104881 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:29.104915 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:29.106554 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:29.107162 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:29.107169 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:29.107262 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:56:29.107780 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:29.107795 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.108869 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.108874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:29.110532 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:29.110544 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.111320 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.111329 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:29.111545 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:29.112234 (+ 689us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:56:29.112237 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:56:29.112279 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:56:29.112372 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:56:29.112696 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:56:29.112707 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:56:29.112715 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 00:56:29.112854 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 00:56:29.113204 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":262,"threads_started":1}
I20260810 00:56:29.115473 9031 heartbeater.cc:347] Connected to a master server at 127.10.97.126:34487
I20260810 00:56:29.115746 9031 heartbeater.cc:464] Registering TS with master...
I20260810 00:56:29.116453 9031 heartbeater.cc:511] Master 127.10.97.126:34487 requested a full tablet report, sending...
I20260810 00:56:29.117803 8546 ts_manager.cc:194] Registered new tserver with Master: ce2b06d499d9423185100b7929d11eca (127.10.97.66:46867)
I20260810 00:56:29.118772 8546 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.97.66:49901
W20260810 00:56:29.428292 9037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:29.428542 9037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:29.428675 9037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:29.435453 9037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:29.435545 9037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:29.435688 9037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.97.67
I20260810 00:56:29.444159 9037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:45145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.97.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.97.67
--webserver_port=0
--tserver_master_addrs=127.10.97.126:34487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.97.67
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:29.445231 9037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:29.446852 9037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:29.458721 9042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:29.458835 9050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:29.458715 9047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:29.459439 9037 server_base.cc:1034] running on GCE node
I20260810 00:56:29.460044 9037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:29.460876 9037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:29.462073 9037 hybrid_clock.cc:648] HybridClock initialized: now 1786323389462033 us; error 29 us; skew 500 ppm
I20260810 00:56:29.462354 9037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:56:29.464442 9037 webserver.cc:460] Webserver started at http://127.10.97.67:42819/ using document root <none> and password file <none>
I20260810 00:56:29.465075 9037 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:29.465165 9037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:29.465412 9037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:29.468112 9037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-2/data/instance:
uuid: "9eebc92ce6a24a3eb2717e55ff915ad7"
format_stamp: "Formatted at 2026-08-10 00:56:29 on dist-test-slave-jlzn"
server_key: "9bf29e6f746eff03f447acae8bf2696c"
server_key_iv: "d7f06c48371ad7941394f2cb4c0c8de2"
server_key_version: "encryptionkey@0"
I20260810 00:56:29.468830 9037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance:
uuid: "9eebc92ce6a24a3eb2717e55ff915ad7"
format_stamp: "Formatted at 2026-08-10 00:56:29 on dist-test-slave-jlzn"
server_key: "9bf29e6f746eff03f447acae8bf2696c"
server_key_iv: "d7f06c48371ad7941394f2cb4c0c8de2"
server_key_version: "encryptionkey@0"
I20260810 00:56:29.473454 9037 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:56:29.476531 9061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:29.477679 9037 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:56:29.477818 9037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-2/wal
uuid: "9eebc92ce6a24a3eb2717e55ff915ad7"
format_stamp: "Formatted at 2026-08-10 00:56:29 on dist-test-slave-jlzn"
server_key: "9bf29e6f746eff03f447acae8bf2696c"
server_key_iv: "d7f06c48371ad7941394f2cb4c0c8de2"
server_key_version: "encryptionkey@0"
I20260810 00:56:29.478005 9037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:29.494838 9037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:29.495514 9037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:29.495790 9037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:29.496665 9037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:56:29.497952 9037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:56:29.498059 9037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:29.498157 9037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:56:29.498226 9037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:29.527436 9037 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.67:37355
I20260810 00:56:29.527490 9234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.67:37355 every 8 connection(s)
I20260810 00:56:29.528590 9037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-2/data/info.pb
I20260810 00:56:29.530112 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 9037
I20260810 00:56:29.530288 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-2/wal/instance
I20260810 00:56:29.530804 10629 external_mini_cluster.cc:1428] Setting key b1d8b4455e44d529de6d8684a1d84346
I20260810 00:56:29.539772 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.530979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.67:51473 (local address 127.10.97.126:34487)
0810 00:56:29.531192 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:29.531198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:29.532655 (+ 1457us) server_negotiation.cc:407] Connection header received
0810 00:56:29.534440 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:29.534445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:29.534509 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:29.534585 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:29.535468 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.536202 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.537613 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.537991 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.538073 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:29.539196 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:29.539200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:29.539203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:29.539219 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:29.539250 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:29.539261 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:29.539323 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:29.539480 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 00:56:29.539550 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 00:56:29.540655 9239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.531409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:34487 (local address 127.10.97.67:51473)
0810 00:56:29.532493 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:29.532546 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:29.534222 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:29.534758 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:29.534765 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:29.534841 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:56:29.535338 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:29.535353 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.536328 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.536332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:29.537488 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:29.537500 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.538141 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.538145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:29.538329 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:29.538907 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 00:56:29.538910 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:56:29.538943 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 00:56:29.539012 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:56:29.539335 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:56:29.539341 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:56:29.539346 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 00:56:29.539466 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 00:56:29.539728 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":417,"threads_started":1}
I20260810 00:56:29.541777 9235 heartbeater.cc:347] Connected to a master server at 127.10.97.126:34487
I20260810 00:56:29.542024 9235 heartbeater.cc:464] Registering TS with master...
I20260810 00:56:29.542681 9235 heartbeater.cc:511] Master 127.10.97.126:34487 requested a full tablet report, sending...
I20260810 00:56:29.543928 8546 ts_manager.cc:194] Registered new tserver with Master: 9eebc92ce6a24a3eb2717e55ff915ad7 (127.10.97.67:37355)
I20260810 00:56:29.544883 8546 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.97.67:51473
I20260810 00:56:29.545722 10629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 00:56:29.554366 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.549411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53008 (local address 127.10.97.126:34487)
0810 00:56:29.549571 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:29.549577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:29.549934 (+ 357us) server_negotiation.cc:407] Connection header received
0810 00:56:29.550051 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:29.550055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:29.550119 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:29.550189 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:29.550564 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.551243 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.552542 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.552898 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.552984 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:29.553769 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:29.553774 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:29.553777 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:29.553793 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:29.553820 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:29.553831 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:29.553896 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:29.553982 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 00:56:29.554069 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 00:56:29.557983 8545 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:56:29.564604 8545 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:56:29.596647 9252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.575365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:46121 (local address 127.0.0.1:38578)
0810 00:56:29.576370 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:29.576411 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:29.576580 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:29.589773 (+ 13193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:29.589781 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:29.589811 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:29.590195 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:29.590209 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.592027 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.592033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:29.594200 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:29.594217 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.595712 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.595718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:29.595942 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:29.596000 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 00:56:29.596103 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 00:56:29.596278 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"mutex_wait_us":40,"thread_start_us":573,"threads_started":1}
I20260810 00:56:29.596832 9261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.576400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53564 (local address 127.10.97.67:37355)
0810 00:56:29.589636 (+ 13236us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:29.589643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:29.589673 (+ 30us) server_negotiation.cc:407] Connection header received
0810 00:56:29.589760 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:29.589766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:29.589962 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:29.590152 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:29.590916 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.592068 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.594594 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.595783 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.595933 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:29.596206 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:29.596351 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 00:56:29.596503 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13080,"thread_start_us":12931,"threads_started":1}
I20260810 00:56:29.597005 9260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.575728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:37355 (local address 127.0.0.1:53564)
0810 00:56:29.588490 (+ 12762us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:29.588509 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:29.588628 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:29.590384 (+ 1756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:29.590389 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:29.590413 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:29.590703 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:29.590714 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.592235 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.592240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:29.594393 (+ 2153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:29.594408 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.596009 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.596016 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:29.596186 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:29.596218 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 00:56:29.596303 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 00:56:29.596419 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12634,"mutex_wait_us":1,"thread_start_us":12491,"threads_started":1}
I20260810 00:56:29.597862 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.576691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:46867 (local address 127.0.0.1:50182)
0810 00:56:29.589929 (+ 13238us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:29.589943 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:29.590067 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:29.590714 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:29.590718 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:29.590745 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:29.591079 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:29.591091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.592995 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.593000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:29.595086 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:29.595101 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.597154 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.597160 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:29.597322 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:29.597352 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 00:56:29.597447 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 00:56:29.597558 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13133,"mutex_wait_us":40,"thread_start_us":12971,"threads_started":1}
I20260810 00:56:29.597975 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.575669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38578 (local address 127.10.97.65:46121)
0810 00:56:29.589095 (+ 13426us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:29.589102 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:29.589129 (+ 27us) server_negotiation.cc:407] Connection header received
0810 00:56:29.589225 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:29.589233 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:29.589496 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:29.590266 (+ 770us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:29.590493 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.591889 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.594414 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.595489 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.597199 (+ 1710us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:29.597423 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:29.597490 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 00:56:29.597652 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13282,"thread_start_us":13172,"threads_started":1}
I20260810 00:56:29.598021 9264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.577224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50182 (local address 127.10.97.66:46867)
0810 00:56:29.587926 (+ 10702us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:29.587935 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:29.590047 (+ 2112us) server_negotiation.cc:407] Connection header received
0810 00:56:29.590234 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:29.590241 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:29.590436 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:29.590647 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:29.591272 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.592836 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.595279 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.596662 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.596789 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:29.597052 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:29.597506 (+ 454us) server_negotiation.cc:299] Negotiation successful
0810 00:56:29.597663 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10501,"thread_start_us":10695,"threads_started":1}
I20260810 00:56:29.600754 9134 tablet_service.cc:1467] Processing CreateTablet for tablet 4e38ef73f9c24b9dabfdd8066faf0b37 (DEFAULT_TABLE table=test-table [id=edd8a56813e54064bd2f4e7b7e4c1f06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:56:29.601312 8725 tablet_service.cc:1467] Processing CreateTablet for tablet 4e38ef73f9c24b9dabfdd8066faf0b37 (DEFAULT_TABLE table=test-table [id=edd8a56813e54064bd2f4e7b7e4c1f06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:56:29.602147 9134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e38ef73f9c24b9dabfdd8066faf0b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:29.602564 8725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e38ef73f9c24b9dabfdd8066faf0b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:29.608958 8931 tablet_service.cc:1467] Processing CreateTablet for tablet 4e38ef73f9c24b9dabfdd8066faf0b37 (DEFAULT_TABLE table=test-table [id=edd8a56813e54064bd2f4e7b7e4c1f06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:56:29.610306 8931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e38ef73f9c24b9dabfdd8066faf0b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:29.613299 9266 tablet_bootstrap.cc:492] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991: Bootstrap starting.
I20260810 00:56:29.615031 9267 tablet_bootstrap.cc:492] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7: Bootstrap starting.
I20260810 00:56:29.617813 9266 tablet_bootstrap.cc:654] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:29.618418 9267 tablet_bootstrap.cc:654] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:29.619022 9266 log.cc:824] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:29.619583 9267 log.cc:824] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:29.621886 9266 tablet_bootstrap.cc:492] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991: No bootstrap required, opened a new log
I20260810 00:56:29.622146 9266 ts_tablet_manager.cc:1397] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 00:56:29.622458 9267 tablet_bootstrap.cc:492] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7: No bootstrap required, opened a new log
I20260810 00:56:29.622727 9267 ts_tablet_manager.cc:1397] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 00:56:29.623291 9272 tablet_bootstrap.cc:492] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca: Bootstrap starting.
I20260810 00:56:29.626271 9272 tablet_bootstrap.cc:654] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:29.627712 9272 log.cc:824] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:29.630450 9272 tablet_bootstrap.cc:492] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca: No bootstrap required, opened a new log
I20260810 00:56:29.630291 9267 raft_consensus.cc:348] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc01db279414490a0f2487e9859f991" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 46121 } } peers { permanent_uuid: "9eebc92ce6a24a3eb2717e55ff915ad7" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37355 } } peers { permanent_uuid: "ce2b06d499d9423185100b7929d11eca" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46867 } }
I20260810 00:56:29.630620 9267 raft_consensus.cc:374] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:29.630692 9272 ts_tablet_manager.cc:1397] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260810 00:56:29.630719 9267 raft_consensus.cc:729] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eebc92ce6a24a3eb2717e55ff915ad7, State: Initialized, Role: FOLLOWER
I20260810 00:56:29.631239 9267 consensus_queue.cc:260] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc01db279414490a0f2487e9859f991" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 46121 } } peers { permanent_uuid: "9eebc92ce6a24a3eb2717e55ff915ad7" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37355 } } peers { permanent_uuid: "ce2b06d499d9423185100b7929d11eca" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46867 } }
I20260810 00:56:29.632238 9235 heartbeater.cc:503] Master 127.10.97.126:34487 was elected leader, sending a full tablet report...
I20260810 00:56:29.632887 9267 ts_tablet_manager.cc:1428] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260810 00:56:29.631773 9266 raft_consensus.cc:348] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc01db279414490a0f2487e9859f991" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 46121 } } peers { permanent_uuid: "9eebc92ce6a24a3eb2717e55ff915ad7" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37355 } } peers { permanent_uuid: "ce2b06d499d9423185100b7929d11eca" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46867 } }
I20260810 00:56:29.633572 9266 raft_consensus.cc:374] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:29.633661 9266 raft_consensus.cc:729] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cc01db279414490a0f2487e9859f991, State: Initialized, Role: FOLLOWER
I20260810 00:56:29.634179 9266 consensus_queue.cc:260] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc01db279414490a0f2487e9859f991" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 46121 } } peers { permanent_uuid: "9eebc92ce6a24a3eb2717e55ff915ad7" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37355 } } peers { permanent_uuid: "ce2b06d499d9423185100b7929d11eca" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46867 } }
I20260810 00:56:29.635728 9266 ts_tablet_manager.cc:1428] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991: Time spent starting tablet: real 0.013s user 0.007s sys 0.006s
I20260810 00:56:29.635869 8826 heartbeater.cc:503] Master 127.10.97.126:34487 was elected leader, sending a full tablet report...
I20260810 00:56:29.639483 9272 raft_consensus.cc:348] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc01db279414490a0f2487e9859f991" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 46121 } } peers { permanent_uuid: "9eebc92ce6a24a3eb2717e55ff915ad7" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37355 } } peers { permanent_uuid: "ce2b06d499d9423185100b7929d11eca" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46867 } }
I20260810 00:56:29.639885 9272 raft_consensus.cc:374] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:29.639983 9272 raft_consensus.cc:729] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce2b06d499d9423185100b7929d11eca, State: Initialized, Role: FOLLOWER
I20260810 00:56:29.640465 9272 consensus_queue.cc:260] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc01db279414490a0f2487e9859f991" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 46121 } } peers { permanent_uuid: "9eebc92ce6a24a3eb2717e55ff915ad7" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37355 } } peers { permanent_uuid: "ce2b06d499d9423185100b7929d11eca" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46867 } }
I20260810 00:56:29.641309 9031 heartbeater.cc:503] Master 127.10.97.126:34487 was elected leader, sending a full tablet report...
I20260810 00:56:29.641649 9272 ts_tablet_manager.cc:1428] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260810 00:56:29.655414 9277 raft_consensus.cc:482] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:56:29.655663 9277 raft_consensus.cc:504] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc01db279414490a0f2487e9859f991" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 46121 } } peers { permanent_uuid: "9eebc92ce6a24a3eb2717e55ff915ad7" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37355 } } peers { permanent_uuid: "ce2b06d499d9423185100b7929d11eca" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46867 } }
I20260810 00:56:29.657085 9277 leader_election.cc:288] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9eebc92ce6a24a3eb2717e55ff915ad7 (127.10.97.67:37355), ce2b06d499d9423185100b7929d11eca (127.10.97.66:46867)
I20260810 00:56:29.665236 9261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.657357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:44807 (local address 127.10.97.67:37355)
0810 00:56:29.657516 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:29.657522 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:29.658251 (+ 729us) server_negotiation.cc:407] Connection header received
0810 00:56:29.658385 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:29.658390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:29.658451 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:29.658524 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:29.659287 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.660390 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.663256 (+ 2866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.664458 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.664633 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:29.664701 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:29.664968 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 00:56:29.665057 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 00:56:29.665400 9280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.657409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.67:37355 (local address 127.10.97.65:44807)
0810 00:56:29.658169 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:29.658183 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:29.658282 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:29.658547 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:29.658551 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:29.658568 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:29.659131 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:29.659145 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.660506 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.660511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:29.663061 (+ 2550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:29.663071 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.664650 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.664661 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:29.664815 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:29.664850 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 00:56:29.664944 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 00:56:29.665070 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":192,"thread_start_us":469,"threads_started":1}
I20260810 00:56:29.665762 9264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.657558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.97.65:35419 (local address 127.10.97.66:46867)
0810 00:56:29.657731 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:29.657737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:29.658148 (+ 411us) server_negotiation.cc:407] Connection header received
0810 00:56:29.658341 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:29.658345 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:29.658415 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:29.658509 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 00:56:29.659308 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.660390 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.663256 (+ 2866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.664467 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.664664 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:29.664734 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:29.665484 (+ 750us) server_negotiation.cc:299] Negotiation successful
0810 00:56:29.665603 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 00:56:29.665870 9279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.657297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.66:46867 (local address 127.10.97.65:35419)
0810 00:56:29.658044 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:29.658063 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:56:29.658211 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:56:29.658524 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:56:29.658536 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:56:29.658562 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 00:56:29.659134 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:29.659145 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.660509 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.660514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:29.663051 (+ 2537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:56:29.663067 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.665045 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.665053 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:56:29.665231 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:29.665260 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 00:56:29.665457 (+ 197us) client_negotiation.cc:240] Negotiation successful
0810 00:56:29.665586 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":163,"thread_start_us":395,"threads_started":1}
I20260810 00:56:29.666313 9164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e38ef73f9c24b9dabfdd8066faf0b37" candidate_uuid: "4cc01db279414490a0f2487e9859f991" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9eebc92ce6a24a3eb2717e55ff915ad7" is_pre_election: true
I20260810 00:56:29.666646 8966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e38ef73f9c24b9dabfdd8066faf0b37" candidate_uuid: "4cc01db279414490a0f2487e9859f991" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce2b06d499d9423185100b7929d11eca" is_pre_election: true
I20260810 00:56:29.666734 9164 raft_consensus.cc:2454] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cc01db279414490a0f2487e9859f991 in term 0.
I20260810 00:56:29.667029 8966 raft_consensus.cc:2454] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cc01db279414490a0f2487e9859f991 in term 0.
I20260810 00:56:29.667318 8662 leader_election.cc:302] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cc01db279414490a0f2487e9859f991, 9eebc92ce6a24a3eb2717e55ff915ad7; no voters:
I20260810 00:56:29.667677 9277 raft_consensus.cc:2781] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:56:29.667872 9277 raft_consensus.cc:482] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:56:29.667974 9277 raft_consensus.cc:3037] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:29.670244 9277 raft_consensus.cc:504] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc01db279414490a0f2487e9859f991" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 46121 } } peers { permanent_uuid: "9eebc92ce6a24a3eb2717e55ff915ad7" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37355 } } peers { permanent_uuid: "ce2b06d499d9423185100b7929d11eca" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46867 } }
I20260810 00:56:29.670806 9277 leader_election.cc:288] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [CANDIDATE]: Term 1 election: Requested vote from peers 9eebc92ce6a24a3eb2717e55ff915ad7 (127.10.97.67:37355), ce2b06d499d9423185100b7929d11eca (127.10.97.66:46867)
I20260810 00:56:29.671150 9164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e38ef73f9c24b9dabfdd8066faf0b37" candidate_uuid: "4cc01db279414490a0f2487e9859f991" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9eebc92ce6a24a3eb2717e55ff915ad7"
W20260810 00:56:29.671201 8828 tablet.cc:2369] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:56:29.671259 8966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e38ef73f9c24b9dabfdd8066faf0b37" candidate_uuid: "4cc01db279414490a0f2487e9859f991" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce2b06d499d9423185100b7929d11eca"
I20260810 00:56:29.671314 9164 raft_consensus.cc:3037] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:29.671439 8966 raft_consensus.cc:3037] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:29.673666 8966 raft_consensus.cc:2454] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cc01db279414490a0f2487e9859f991 in term 1.
I20260810 00:56:29.674074 8658 leader_election.cc:302] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cc01db279414490a0f2487e9859f991, ce2b06d499d9423185100b7929d11eca; no voters:
I20260810 00:56:29.674342 9164 raft_consensus.cc:2454] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cc01db279414490a0f2487e9859f991 in term 1.
I20260810 00:56:29.674378 9277 raft_consensus.cc:2781] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:29.674868 9277 raft_consensus.cc:686] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [term 1 LEADER]: Becoming Leader. State: Replica: 4cc01db279414490a0f2487e9859f991, State: Running, Role: LEADER
I20260810 00:56:29.675297 9277 consensus_queue.cc:237] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc01db279414490a0f2487e9859f991" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 46121 } } peers { permanent_uuid: "9eebc92ce6a24a3eb2717e55ff915ad7" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37355 } } peers { permanent_uuid: "ce2b06d499d9423185100b7929d11eca" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46867 } }
I20260810 00:56:29.679464 8546 catalog_manager.cc:5495] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cc01db279414490a0f2487e9859f991 (127.10.97.65). New cstate: current_term: 1 leader_uuid: "4cc01db279414490a0f2487e9859f991" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc01db279414490a0f2487e9859f991" member_type: VOTER last_known_addr { host: "127.10.97.65" port: 46121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9eebc92ce6a24a3eb2717e55ff915ad7" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce2b06d499d9423185100b7929d11eca" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46867 } health_report { overall_health: UNKNOWN } } }
I20260810 00:56:29.708076 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.703395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38588 (local address 127.10.97.65:46121)
0810 00:56:29.703546 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:29.703550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:29.703569 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:56:29.703737 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:29.703741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:29.703837 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:29.703944 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:56:29.704216 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.704856 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.706165 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.706620 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.706720 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:29.707140 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:56:29.707591 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:56:29.707690 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:29.707760 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 00:56:29.707842 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 00:56:29.715466 9164 raft_consensus.cc:1261] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7 [term 1 FOLLOWER]: Refusing update from remote peer 4cc01db279414490a0f2487e9859f991: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:56:29.715462 8966 raft_consensus.cc:1261] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca [term 1 FOLLOWER]: Refusing update from remote peer 4cc01db279414490a0f2487e9859f991: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:56:29.716406 9281 consensus_queue.cc:1035] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9eebc92ce6a24a3eb2717e55ff915ad7" member_type: VOTER last_known_addr { host: "127.10.97.67" port: 37355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:56:29.716745 9277 consensus_queue.cc:1035] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce2b06d499d9423185100b7929d11eca" member_type: VOTER last_known_addr { host: "127.10.97.66" port: 46867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:56:29.732967 9292 mvcc.cc:205] Tried to move back new op lower bound from 7316780604264624128 to 7316780604113395712. Current Snapshot: MvccSnapshot[applied={T|T < 7316780604264624128}]
I20260810 00:56:29.741278 9294 mvcc.cc:205] Tried to move back new op lower bound from 7316780604264624128 to 7316780604113395712. Current Snapshot: MvccSnapshot[applied={T|T < 7316780604264624128}]
I20260810 00:56:29.748698 8546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:56:29.748925 8546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 00:56:29.752166 8546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7bff0cc45114bd0bded7c5ef7f7d6ed: Sending DeleteTablet for 3 replicas of tablet 4e38ef73f9c24b9dabfdd8066faf0b37
I20260810 00:56:29.753196 9134 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e38ef73f9c24b9dabfdd8066faf0b37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:56:29 UTC) from {username='slave'} at 127.0.0.1:53564
I20260810 00:56:29.753209 8931 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e38ef73f9c24b9dabfdd8066faf0b37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:56:29 UTC) from {username='slave'} at 127.0.0.1:50182
I20260810 00:56:29.753273 8725 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e38ef73f9c24b9dabfdd8066faf0b37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:56:29 UTC) from {username='slave'} at 127.0.0.1:38578
I20260810 00:56:29.760085 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 8636
I20260810 00:56:29.773914 9305 tablet_replica.cc:330] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca: stopping tablet replica
I20260810 00:56:29.774557 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.761592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38602 (local address 127.10.97.65:46121)
0810 00:56:29.761748 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:29.761754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:29.761844 (+ 90us) server_negotiation.cc:407] Connection header received
0810 00:56:29.762487 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:29.762506 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:29.762613 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:29.762700 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:29.763949 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.765198 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.768877 (+ 3679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.769612 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.769714 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:29.772177 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:29.772193 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:29.772212 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:29.773006 (+ 794us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:29.773070 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:29.773098 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:29.773179 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:29.774047 (+ 868us) server_negotiation.cc:299] Negotiation successful
0810 00:56:29.774156 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 00:56:29.775004 9308 tablet_replica.cc:330] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7: stopping tablet replica
I20260810 00:56:29.774966 9305 raft_consensus.cc:2229] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:56:29.774164 9311 tablet_replica.cc:330] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991: stopping tablet replica
I20260810 00:56:29.775430 8818 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:29.775552 9308 raft_consensus.cc:2229] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:56:29.776070 9305 raft_consensus.cc:2258] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:56:29.776070 9308 raft_consensus.cc:2258] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:56:29.778098 9308 ts_tablet_manager.cc:1905] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:56:29.778098 9305 ts_tablet_manager.cc:1905] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:56:29.785270 9308 ts_tablet_manager.cc:1918] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:56:29.785267 9305 ts_tablet_manager.cc:1918] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 00:56:29.785429 9305 log.cc:1196] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-1/wal/wals/4e38ef73f9c24b9dabfdd8066faf0b37
I20260810 00:56:29.785429 9308 log.cc:1196] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7: Deleting WAL directory at /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786323077328248-10629-0/minicluster-data/ts-2/wal/wals/4e38ef73f9c24b9dabfdd8066faf0b37
I20260810 00:56:29.785938 9308 ts_tablet_manager.cc:1939] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 9eebc92ce6a24a3eb2717e55ff915ad7: Deleting consensus metadata
I20260810 00:56:29.785938 9305 ts_tablet_manager.cc:1939] T 4e38ef73f9c24b9dabfdd8066faf0b37 P ce2b06d499d9423185100b7929d11eca: Deleting consensus metadata
I20260810 00:56:29.787619 8531 catalog_manager.cc:4841] TS ce2b06d499d9423185100b7929d11eca (127.10.97.66:46867): tablet 4e38ef73f9c24b9dabfdd8066faf0b37 (table test-table [id=edd8a56813e54064bd2f4e7b7e4c1f06]) successfully deleted
I20260810 00:56:29.787662 8532 catalog_manager.cc:4841] TS 9eebc92ce6a24a3eb2717e55ff915ad7 (127.10.97.67:37355): tablet 4e38ef73f9c24b9dabfdd8066faf0b37 (table test-table [id=edd8a56813e54064bd2f4e7b7e4c1f06]) successfully deleted
I20260810 00:56:29.775544 9311 raft_consensus.cc:2229] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:56:29.874647 9311 raft_consensus.cc:2258] T 4e38ef73f9c24b9dabfdd8066faf0b37 P 4cc01db279414490a0f2487e9859f991 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:56:29.874775 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 8636
W20260810 00:56:29.896991 8531 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.97.65:46121 (error 108)
I20260810 00:56:29.897224 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 8831
I20260810 00:56:29.898872 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.898420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:46121 (local address 127.0.0.1:38614)
0810 00:56:29.898575 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:29.898699 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:46121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 00:56:29.899128 8531 catalog_manager.cc:4584] TS 4cc01db279414490a0f2487e9859f991 (127.10.97.65:46121): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4e38ef73f9c24b9dabfdd8066faf0b37: Network error: Client connection negotiation failed: client connection to 127.10.97.65:46121: connect: Connection refused (error 111)
I20260810 00:56:29.903824 9264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.898053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50188 (local address 127.10.97.66:46867)
0810 00:56:29.898236 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:29.898242 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:29.898264 (+ 22us) server_negotiation.cc:407] Connection header received
0810 00:56:29.898426 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:29.898432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:29.898556 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:29.898654 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:29.899110 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.899926 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.901110 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:29.901503 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:29.901610 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:29.902537 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:29.902546 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:29.902560 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:29.903108 (+ 548us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:29.903155 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:29.903174 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:29.903268 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:29.903400 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 00:56:29.903508 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 00:56:29.904479 9026 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:29.952780 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.952319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:46121 (local address 127.0.0.1:38618)
0810 00:56:29.952518 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:29.952614 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:46121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260810 00:56:29.953768 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:29.953411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:46121 (local address 127.0.0.1:38632)
0810 00:56:29.953560 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:29.953632 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:46121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 00:56:29.990060 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 8831
I20260810 00:56:30.013970 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:30.013515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:46121 (local address 127.0.0.1:38640)
0810 00:56:30.013712 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:30.013793 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:46121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 00:56:30.014830 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 9037
I20260810 00:56:30.014961 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:30.014596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:46121 (local address 127.0.0.1:38646)
0810 00:56:30.014745 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:30.014820 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:46121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 00:56:30.021003 9261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:30.015516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53576 (local address 127.10.97.67:37355)
0810 00:56:30.015669 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:30.015675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:30.015708 (+ 33us) server_negotiation.cc:407] Connection header received
0810 00:56:30.015905 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:30.015909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:30.016007 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:30.016088 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:30.016433 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:30.017139 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:30.018390 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:30.018779 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:30.018867 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:30.019708 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:30.019717 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:30.019732 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:30.020432 (+ 700us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:30.020485 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:30.020506 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:30.020590 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:30.020683 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 00:56:30.020764 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 00:56:30.021575 9229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:30.095397 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:30.094875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:46121 (local address 127.0.0.1:38662)
0810 00:56:30.095087 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:30.095208 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:46121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 00:56:30.096472 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:30.096035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.65:46121 (local address 127.0.0.1:38672)
0810 00:56:30.096251 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 00:56:30.096339 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.65:46121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260810 00:56:30.107262 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 9037
I20260810 00:56:30.128551 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 8509
I20260810 00:56:30.129503 8595 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:30.233666 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 8509
2026-08-10T00:56:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2549 ms)
[----------] 6 tests from EncryptionPolicyTest (21024 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:56:30 dist-test-slave-jlzn krb5kdc[9329](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:56:30 dist-test-slave-jlzn krb5kdc[9329](info): set up 2 sockets
Aug 10 00:56:30 dist-test-slave-jlzn krb5kdc[9329](info): commencing operation
krb5kdc: starting...
W20260810 00:56:31.434381 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:56:31 dist-test-slave-jlzn krb5kdc[9329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:56:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 00:56:31.681077 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37115
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:31.996030 9344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:31.996302 9344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:31.996443 9344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:32.003218 9344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:32.003326 9344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:32.003404 9344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:32.003489 9344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:32.003551 9344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:32.012717 9344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37115
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:32.013787 9344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:32.014958 9344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:32.025676 9349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:32.025718 9350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:32.025645 9353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:32.026402 9344 server_base.cc:1034] running on GCE node
I20260810 00:56:32.027094 9344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:32.028343 9344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:32.029541 9344 hybrid_clock.cc:648] HybridClock initialized: now 1786323392029499 us; error 30 us; skew 500 ppm
I20260810 00:56:32.029847 9344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:32 dist-test-slave-jlzn krb5kdc[9329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323392, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:32.038688 9344 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 00:56:32.040055 9344 webserver.cc:460] Webserver started at http://127.0.0.1:38759/ using document root <none> and password file <none>
I20260810 00:56:32.040655 9344 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:32.040776 9344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:32.041044 9344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:32.043769 9344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "096851b74df442d3bc6b74d2d416dc11"
format_stamp: "Formatted at 2026-08-10 00:56:32 on dist-test-slave-jlzn"
server_key: "df16e394cf0a2cb4473fb31548a24f59"
server_key_iv: "d4a0d6ba4f03d3055ea96e412cc8f5ee"
server_key_version: "encryptionkey@0"
I20260810 00:56:32.044523 9344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "096851b74df442d3bc6b74d2d416dc11"
format_stamp: "Formatted at 2026-08-10 00:56:32 on dist-test-slave-jlzn"
server_key: "df16e394cf0a2cb4473fb31548a24f59"
server_key_iv: "d4a0d6ba4f03d3055ea96e412cc8f5ee"
server_key_version: "encryptionkey@0"
I20260810 00:56:32.049126 9344 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260810 00:56:32.052279 9364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:32.053439 9344 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:56:32.053606 9344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "096851b74df442d3bc6b74d2d416dc11"
format_stamp: "Formatted at 2026-08-10 00:56:32 on dist-test-slave-jlzn"
server_key: "df16e394cf0a2cb4473fb31548a24f59"
server_key_iv: "d4a0d6ba4f03d3055ea96e412cc8f5ee"
server_key_version: "encryptionkey@0"
I20260810 00:56:32.053772 9344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:32.084065 9344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:32.087505 9344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:32.087750 9344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:32.104301 9344 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37115
I20260810 00:56:32.104334 9442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37115 every 8 connection(s)
I20260810 00:56:32.105494 9344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:32.108886 9443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:32.110301 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 9344
I20260810 00:56:32.110526 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:32.111402 10629 external_mini_cluster.cc:1428] Setting key f53cc9bee520069e6d15993f62886573
I20260810 00:56:32.122670 9443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11: Bootstrap starting.
Aug 10 00:56:32 dist-test-slave-jlzn krb5kdc[9329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 00:56:32.126271 9443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:32.127398 9443 log.cc:824] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:32.130211 9443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11: No bootstrap required, opened a new log
I20260810 00:56:32.133980 9449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:32.113316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51085 (local address 127.0.0.1:37115)
0810 00:56:32.114108 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:32.114118 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:32.114160 (+ 42us) server_negotiation.cc:407] Connection header received
0810 00:56:32.116035 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:32.116059 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:32.116416 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:32.116702 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:32.118419 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:32.119718 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:32.121480 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:32.122102 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:32.122229 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:32.126896 (+ 4667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:32.126903 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:32.126916 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:32.126949 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:32.129964 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:32.130821 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:32.130827 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:32.130832 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:32.131006 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:32.131481 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:32.131484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:32.131486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:32.132031 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:32.132297 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:32.132751 (+ 454us) server_negotiation.cc:299] Negotiation successful
0810 00:56:32.133006 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":367,"threads_started":1}
I20260810 00:56:32.137526 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:56:32.138494 9443 raft_consensus.cc:348] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "096851b74df442d3bc6b74d2d416dc11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37115 } }
I20260810 00:56:32.138859 9443 raft_consensus.cc:374] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:32.138948 9443 raft_consensus.cc:729] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 096851b74df442d3bc6b74d2d416dc11, State: Initialized, Role: FOLLOWER
I20260810 00:56:32.139500 9443 consensus_queue.cc:260] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "096851b74df442d3bc6b74d2d416dc11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37115 } }
I20260810 00:56:32.139760 9443 raft_consensus.cc:388] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:32.139916 9443 raft_consensus.cc:482] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:32.140084 9443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:32.142932 9443 raft_consensus.cc:504] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "096851b74df442d3bc6b74d2d416dc11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37115 } }
I20260810 00:56:32.143404 9443 leader_election.cc:302] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 096851b74df442d3bc6b74d2d416dc11; no voters:
I20260810 00:56:32.144025 9443 leader_election.cc:288] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:32.144171 9461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:32.144804 9461 raft_consensus.cc:686] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11 [term 1 LEADER]: Becoming Leader. State: Replica: 096851b74df442d3bc6b74d2d416dc11, State: Running, Role: LEADER
I20260810 00:56:32.145406 9461 consensus_queue.cc:237] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "096851b74df442d3bc6b74d2d416dc11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37115 } }
I20260810 00:56:32.147054 9461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 096851b74df442d3bc6b74d2d416dc11. Latest consensus state: current_term: 1 leader_uuid: "096851b74df442d3bc6b74d2d416dc11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "096851b74df442d3bc6b74d2d416dc11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37115 } } }
I20260810 00:56:32.148550 9461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:32.148226 9464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "096851b74df442d3bc6b74d2d416dc11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "096851b74df442d3bc6b74d2d416dc11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37115 } } }
I20260810 00:56:32.148927 9464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:32.152253 9443 sys_catalog.cc:564] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:32.152441 9468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:32.159974 9449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:32.142819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34885 (local address 127.0.0.1:37115)
0810 00:56:32.142990 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:32.142996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:32.143458 (+ 462us) server_negotiation.cc:407] Connection header received
0810 00:56:32.143618 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:32.143623 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:32.143700 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:32.143840 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:32.145547 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:32.146767 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:32.148732 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:32.149447 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:32.149598 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:32.152498 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:32.152507 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:32.152512 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:32.152559 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:32.156304 (+ 3745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:32.157223 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:32.157229 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:32.157233 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:32.157396 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:32.157845 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:32.157850 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:32.157854 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:32.158271 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:32.158526 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:32.159377 (+ 851us) server_negotiation.cc:299] Negotiation successful
0810 00:56:32.159555 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 00:56:32.163129 9468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:32.172771 9468 catalog_manager.cc:1347] Generated new cluster ID: a4473f1d5a72410dac1340803e7539e8
I20260810 00:56:32.172861 9468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:32.218472 9468 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:32.219653 9468 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:32.245935 9468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 096851b74df442d3bc6b74d2d416dc11: Generated new TSK 0
I20260810 00:56:32.246694 9468 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:56:32.257977 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 9344
I20260810 00:56:32.259384 9437 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:32.352710 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 9344
W20260810 00:56:32.375677 9327 connection.cc:535] client connection to 127.0.0.1:37115 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37115): Connection reset by peer (error 104)
2026-08-10T00:56:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2113 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:56:32 dist-test-slave-jlzn krb5kdc[9493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:56:32 dist-test-slave-jlzn krb5kdc[9493](info): set up 2 sockets
Aug 10 00:56:32 dist-test-slave-jlzn krb5kdc[9493](info): commencing operation
krb5kdc: starting...
W20260810 00:56:33.543391 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.001s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:56:33 dist-test-slave-jlzn krb5kdc[9493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:56:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 00:56:33.795874 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45517
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:34.110494 9511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:34.110770 9511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:34.110905 9511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:34.117918 9511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:34.118028 9511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:34.118104 9511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:34.118192 9511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:34.118249 9511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:34.127661 9511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45517
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:34.128834 9511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:34.130034 9511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:34.140717 9518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:34.140761 9522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:34.140782 9519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:34.141544 9511 server_base.cc:1034] running on GCE node
I20260810 00:56:34.142266 9511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:34.143492 9511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:34.144683 9511 hybrid_clock.cc:648] HybridClock initialized: now 1786323394144635 us; error 38 us; skew 500 ppm
I20260810 00:56:34.144992 9511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:34 dist-test-slave-jlzn krb5kdc[9493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323394, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:34.147926 9511 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 00:56:34.149223 9511 webserver.cc:460] Webserver started at http://127.0.0.1:43289/ using document root <none> and password file <none>
I20260810 00:56:34.149842 9511 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:34.149959 9511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:34.150239 9511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:34.153082 9511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "e1f281ede39f4d61af63527e084bd333"
format_stamp: "Formatted at 2026-08-10 00:56:34 on dist-test-slave-jlzn"
server_key: "64575c5c774c89c023a15df95f3d417d"
server_key_iv: "e38ffc0f86991ab10a633dd63bf7b59e"
server_key_version: "encryptionkey@0"
I20260810 00:56:34.153827 9511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "e1f281ede39f4d61af63527e084bd333"
format_stamp: "Formatted at 2026-08-10 00:56:34 on dist-test-slave-jlzn"
server_key: "64575c5c774c89c023a15df95f3d417d"
server_key_iv: "e38ffc0f86991ab10a633dd63bf7b59e"
server_key_version: "encryptionkey@0"
I20260810 00:56:34.158597 9511 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260810 00:56:34.161773 9532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:34.162945 9511 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:56:34.163102 9511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "e1f281ede39f4d61af63527e084bd333"
format_stamp: "Formatted at 2026-08-10 00:56:34 on dist-test-slave-jlzn"
server_key: "64575c5c774c89c023a15df95f3d417d"
server_key_iv: "e38ffc0f86991ab10a633dd63bf7b59e"
server_key_version: "encryptionkey@0"
I20260810 00:56:34.163278 9511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:34.195997 9511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:34.199530 9511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:34.199783 9511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:34.216398 9511 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45517
I20260810 00:56:34.216435 9611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45517 every 8 connection(s)
I20260810 00:56:34.217660 9511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:34.220992 9612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:34.227321 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 9511
I20260810 00:56:34.227491 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:34.227941 10629 external_mini_cluster.cc:1428] Setting key 4e7d76765d66a3ea098b77d375176b57
I20260810 00:56:34.230865 9612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333: Bootstrap starting.
I20260810 00:56:34.233916 9612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:34.234998 9612 log.cc:824] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:34.237782 9612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333: No bootstrap required, opened a new log
I20260810 00:56:34.240193 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:34.229473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33025 (local address 127.0.0.1:45517)
0810 00:56:34.230183 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:34.230194 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:34.230233 (+ 39us) server_negotiation.cc:407] Connection header received
0810 00:56:34.231907 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:34.231927 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:34.232309 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:34.232609 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:34.233162 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:34.234434 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:34.236121 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:34.236748 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:34.236889 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:34.238039 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:34.238046 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:34.238058 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:34.238561 (+ 503us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:34.238643 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:34.238667 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:34.238786 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:34.239033 (+ 247us) server_negotiation.cc:299] Negotiation successful
0810 00:56:34.239196 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":328,"threads_started":1}
I20260810 00:56:34.243611 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:56:34.245325 9612 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f281ede39f4d61af63527e084bd333" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45517 } }
I20260810 00:56:34.245635 9612 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:34.245733 9612 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f281ede39f4d61af63527e084bd333, State: Initialized, Role: FOLLOWER
I20260810 00:56:34.246311 9612 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f281ede39f4d61af63527e084bd333" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45517 } }
I20260810 00:56:34.246568 9612 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:34.246714 9612 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:34.246881 9612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:34.249334 9612 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f281ede39f4d61af63527e084bd333" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45517 } }
I20260810 00:56:34.249782 9612 leader_election.cc:302] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1f281ede39f4d61af63527e084bd333; no voters:
I20260810 00:56:34.250360 9612 leader_election.cc:288] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:34.250444 9626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:34.252571 9612 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:34.253906 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:34.247649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54359 (local address 127.0.0.1:45517)
0810 00:56:34.247826 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:34.247832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:34.248329 (+ 497us) server_negotiation.cc:407] Connection header received
0810 00:56:34.248472 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:34.248477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:34.248539 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:34.248616 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:34.249092 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:34.249897 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:34.251440 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:34.251964 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:34.252076 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:34.253192 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:34.253197 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:34.253200 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:34.253215 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:34.253271 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:34.253286 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:34.253379 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:34.253561 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 00:56:34.253607 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 00:56:34.256088 9626 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333 [term 1 LEADER]: Becoming Leader. State: Replica: e1f281ede39f4d61af63527e084bd333, State: Running, Role: LEADER
I20260810 00:56:34.256650 9626 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f281ede39f4d61af63527e084bd333" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45517 } }
I20260810 00:56:34.259496 9627 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1f281ede39f4d61af63527e084bd333" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f281ede39f4d61af63527e084bd333" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45517 } } }
I20260810 00:56:34.259879 9627 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:34.260205 9628 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1f281ede39f4d61af63527e084bd333. Latest consensus state: current_term: 1 leader_uuid: "e1f281ede39f4d61af63527e084bd333" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f281ede39f4d61af63527e084bd333" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45517 } } }
I20260810 00:56:34.260524 9628 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333 [sys.catalog]: This master's current role is: LEADER
W20260810 00:56:34.263979 9648 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 00:56:34.264096 9648 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 00:56:34.264256 9649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:34.266358 9649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:34.274339 9649 catalog_manager.cc:1347] Generated new cluster ID: d1076001b1dc4997889311f458c76999
I20260810 00:56:34.274434 9649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:34.299284 9649 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:34.300596 9649 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:34.311682 9649 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1f281ede39f4d61af63527e084bd333: Generated new TSK 0
I20260810 00:56:34.312602 9649 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:56:34.328238 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 9511
W20260810 00:56:34.329360 9603 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33025
W20260810 00:56:34.330073 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 9511: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:56:34.330235 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 9511
W20260810 00:56:34.356050 9489 connection.cc:535] client connection to 127.0.0.1:45517 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45517): Connection reset by peer (error 104)
2026-08-10T00:56:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1980 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:56:34 dist-test-slave-jlzn krb5kdc[9658](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:56:34 dist-test-slave-jlzn krb5kdc[9658](info): set up 2 sockets
Aug 10 00:56:34 dist-test-slave-jlzn krb5kdc[9658](info): commencing operation
krb5kdc: starting...
W20260810 00:56:35.525241 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:56:35 dist-test-slave-jlzn krb5kdc[9658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:56:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 00:56:35.772290 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40333
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:36.091245 9677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:36.091557 9677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:36.091699 9677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:36.098687 9677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:36.098788 9677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:36.098871 9677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:36.098953 9677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:36.099014 9677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:36.108386 9677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40333
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:36.109539 9677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:36.110706 9677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:36.121621 9689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:36.121570 9685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:36.121662 9687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:36.122313 9677 server_base.cc:1034] running on GCE node
I20260810 00:56:36.123001 9677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:36.124321 9677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:36.125520 9677 hybrid_clock.cc:648] HybridClock initialized: now 1786323396125474 us; error 37 us; skew 500 ppm
I20260810 00:56:36.125830 9677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:36 dist-test-slave-jlzn krb5kdc[9658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323396, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:36.128778 9677 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 00:56:36.130049 9677 webserver.cc:460] Webserver started at http://127.0.0.1:43591/ using document root <none> and password file <none>
I20260810 00:56:36.130636 9677 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:36.130753 9677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:36.131026 9677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:36.133817 9677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "10071ecfeac840cda23741f10d850bc1"
format_stamp: "Formatted at 2026-08-10 00:56:36 on dist-test-slave-jlzn"
server_key: "a1a6f1854ac0ad38718e512baad70c19"
server_key_iv: "7e9ad2dbd858bbdc3fd75d3a2cf96aa8"
server_key_version: "encryptionkey@0"
I20260810 00:56:36.134528 9677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "10071ecfeac840cda23741f10d850bc1"
format_stamp: "Formatted at 2026-08-10 00:56:36 on dist-test-slave-jlzn"
server_key: "a1a6f1854ac0ad38718e512baad70c19"
server_key_iv: "7e9ad2dbd858bbdc3fd75d3a2cf96aa8"
server_key_version: "encryptionkey@0"
I20260810 00:56:36.139447 9677 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 00:56:36.142619 9698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:36.143774 9677 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 00:56:36.143997 9677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "10071ecfeac840cda23741f10d850bc1"
format_stamp: "Formatted at 2026-08-10 00:56:36 on dist-test-slave-jlzn"
server_key: "a1a6f1854ac0ad38718e512baad70c19"
server_key_iv: "7e9ad2dbd858bbdc3fd75d3a2cf96aa8"
server_key_version: "encryptionkey@0"
I20260810 00:56:36.144170 9677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:36.149349 9677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:36.152866 9677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:36.153117 9677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:36.169587 9677 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40333
I20260810 00:56:36.169643 9787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40333 every 8 connection(s)
I20260810 00:56:36.170790 9677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:36.173257 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 9677
I20260810 00:56:36.173424 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:36.173822 10629 external_mini_cluster.cc:1428] Setting key 8b8cdbaf60ea87125ba47b0180fd2633
I20260810 00:56:36.174204 9790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:36.181316 9796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:36.175360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36355 (local address 127.0.0.1:40333)
0810 00:56:36.176135 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:36.176150 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:36.176196 (+ 46us) server_negotiation.cc:407] Connection header received
0810 00:56:36.177695 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:36.177732 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:36.178147 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:36.178547 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:36.179490 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:36.179497 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:36.179514 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:36.179544 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:36.179633 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:36.179661 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:36.179883 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:36.180014 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 00:56:36.180205 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":305,"threads_started":1}
I20260810 00:56:36.184580 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:56:36.188378 9790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1: Bootstrap starting.
I20260810 00:56:36.191691 9790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:36.191720 9796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:36.189556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54553 (local address 127.0.0.1:40333)
0810 00:56:36.189710 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:36.189719 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:36.190146 (+ 427us) server_negotiation.cc:407] Connection header received
0810 00:56:36.190249 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:36.190254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:36.190314 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:36.190410 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:36.191156 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:36.191161 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:36.191165 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:36.191182 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:36.191224 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:36.191242 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:36.191354 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:36.191397 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 00:56:36.191444 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 00:56:36.192899 9790 log.cc:824] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:36.195338 9790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1: No bootstrap required, opened a new log
I20260810 00:56:36.202518 9790 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10071ecfeac840cda23741f10d850bc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40333 } }
I20260810 00:56:36.202831 9790 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:36.202924 9790 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10071ecfeac840cda23741f10d850bc1, State: Initialized, Role: FOLLOWER
I20260810 00:56:36.203451 9790 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10071ecfeac840cda23741f10d850bc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40333 } }
I20260810 00:56:36.203675 9790 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:36.203820 9790 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:36.203975 9790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:36.205927 9790 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10071ecfeac840cda23741f10d850bc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40333 } }
I20260810 00:56:36.206363 9790 leader_election.cc:302] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10071ecfeac840cda23741f10d850bc1; no voters:
I20260810 00:56:36.206900 9790 leader_election.cc:288] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:36.207008 9805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:36.207628 9805 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1 [term 1 LEADER]: Becoming Leader. State: Replica: 10071ecfeac840cda23741f10d850bc1, State: Running, Role: LEADER
I20260810 00:56:36.208174 9805 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10071ecfeac840cda23741f10d850bc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40333 } }
I20260810 00:56:36.208837 9790 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:36.211058 9808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10071ecfeac840cda23741f10d850bc1. Latest consensus state: current_term: 1 leader_uuid: "10071ecfeac840cda23741f10d850bc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10071ecfeac840cda23741f10d850bc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40333 } } }
I20260810 00:56:36.211122 9807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10071ecfeac840cda23741f10d850bc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10071ecfeac840cda23741f10d850bc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40333 } } }
I20260810 00:56:36.211481 9808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:36.211627 9807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:36.212684 9817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:36.216620 9817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:36.234608 9817 catalog_manager.cc:1347] Generated new cluster ID: ae7d0b8af14a4595b49cba661e32a8ef
I20260810 00:56:36.234709 9817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:36.252140 9817 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:36.252524 9817 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:36.261240 9817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10071ecfeac840cda23741f10d850bc1: Generated new TSK 0
I20260810 00:56:36.261909 9817 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:56:36.274435 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 9677
W20260810 00:56:36.275450 9780 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36355
W20260810 00:56:36.276058 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 9677: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:56:36.276201 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 9677
2026-08-10T00:56:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1943 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:56:36 dist-test-slave-jlzn krb5kdc[9833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:56:36 dist-test-slave-jlzn krb5kdc[9833](info): set up 2 sockets
Aug 10 00:56:36 dist-test-slave-jlzn krb5kdc[9833](info): commencing operation
krb5kdc: starting...
W20260810 00:56:37.463594 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:56:37 dist-test-slave-jlzn krb5kdc[9833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:56:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 00:56:37.708227 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46639
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:38.023689 9849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:38.023988 9849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:38.024127 9849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:38.031020 9849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:38.031121 9849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:38.031188 9849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:38.031270 9849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:38.031327 9849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:38.040946 9849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:38.042014 9849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:38.043188 9849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:38.053797 9860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:38.053828 9862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:38.054512 9864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:38.054899 9849 server_base.cc:1034] running on GCE node
I20260810 00:56:38.055598 9849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:38.056862 9849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:38.058072 9849 hybrid_clock.cc:648] HybridClock initialized: now 1786323398058026 us; error 31 us; skew 500 ppm
I20260810 00:56:38.058382 9849 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:38 dist-test-slave-jlzn krb5kdc[9833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323398, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:38.063628 9849 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 00:56:38.065042 9849 webserver.cc:460] Webserver started at http://127.0.0.1:41787/ using document root <none> and password file <none>
I20260810 00:56:38.065639 9849 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:38.065760 9849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:38.066033 9849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:38.068835 9849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "e08e756b2ebb4cb3a2b3f172e274c8f9"
format_stamp: "Formatted at 2026-08-10 00:56:38 on dist-test-slave-jlzn"
server_key: "722437bed104c3bbb76f6ec73b7dc817"
server_key_iv: "a9556c91cd9b95ca5070de5f831f9c81"
server_key_version: "encryptionkey@0"
I20260810 00:56:38.069564 9849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "e08e756b2ebb4cb3a2b3f172e274c8f9"
format_stamp: "Formatted at 2026-08-10 00:56:38 on dist-test-slave-jlzn"
server_key: "722437bed104c3bbb76f6ec73b7dc817"
server_key_iv: "a9556c91cd9b95ca5070de5f831f9c81"
server_key_version: "encryptionkey@0"
I20260810 00:56:38.074258 9849 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260810 00:56:38.077452 9877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:38.078639 9849 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:56:38.078801 9849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "e08e756b2ebb4cb3a2b3f172e274c8f9"
format_stamp: "Formatted at 2026-08-10 00:56:38 on dist-test-slave-jlzn"
server_key: "722437bed104c3bbb76f6ec73b7dc817"
server_key_iv: "a9556c91cd9b95ca5070de5f831f9c81"
server_key_version: "encryptionkey@0"
I20260810 00:56:38.078966 9849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:38.097056 9849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:38.100728 9849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:38.101048 9849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:38.117874 9849 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46639
I20260810 00:56:38.117940 9955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46639 every 8 connection(s)
I20260810 00:56:38.119113 9849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:38.122504 9957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:38.127681 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 9849
I20260810 00:56:38.127871 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:38.128285 10629 external_mini_cluster.cc:1428] Setting key 580e1d94fb2ee9919d4544ed1157e23d
I20260810 00:56:38.133352 9957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9: Bootstrap starting.
I20260810 00:56:38.136762 9957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:38.137853 9957 log.cc:824] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:38.140631 9957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9: No bootstrap required, opened a new log
Aug 10 00:56:38 dist-test-slave-jlzn krb5kdc[9833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 00:56:38.149606 9957 raft_consensus.cc:348] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08e756b2ebb4cb3a2b3f172e274c8f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46639 } }
I20260810 00:56:38.150018 9957 raft_consensus.cc:374] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:38.150141 9957 raft_consensus.cc:729] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e08e756b2ebb4cb3a2b3f172e274c8f9, State: Initialized, Role: FOLLOWER
I20260810 00:56:38.150743 9957 consensus_queue.cc:260] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08e756b2ebb4cb3a2b3f172e274c8f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46639 } }
I20260810 00:56:38.151021 9957 raft_consensus.cc:388] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:38.151176 9957 raft_consensus.cc:482] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:38.151296 9960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:38.130055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42187 (local address 127.0.0.1:46639)
0810 00:56:38.130836 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:38.130850 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:38.130898 (+ 48us) server_negotiation.cc:407] Connection header received
0810 00:56:38.133176 (+ 2278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:38.133202 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:38.133672 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:38.134071 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:38.135359 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:38.137286 (+ 1927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:38.138656 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:38.139473 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:38.139652 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:38.143411 (+ 3759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:38.143422 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:38.143442 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:38.143494 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:38.147241 (+ 3747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:38.147883 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:38.147891 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:38.147899 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:38.148115 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:38.148448 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:38.148453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:38.148456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:38.149115 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:38.149400 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:38.149667 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 00:56:38.150077 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":332,"threads_started":1}
I20260810 00:56:38.151386 9957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:38.153676 9957 raft_consensus.cc:504] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08e756b2ebb4cb3a2b3f172e274c8f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46639 } }
I20260810 00:56:38.154153 9957 leader_election.cc:302] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e08e756b2ebb4cb3a2b3f172e274c8f9; no voters:
I20260810 00:56:38.154696 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:56:38.154740 9957 leader_election.cc:288] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:38.154831 9963 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:38.156842 9957 sys_catalog.cc:564] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:38.161669 9963 raft_consensus.cc:686] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9 [term 1 LEADER]: Becoming Leader. State: Replica: e08e756b2ebb4cb3a2b3f172e274c8f9, State: Running, Role: LEADER
I20260810 00:56:38.162217 9963 consensus_queue.cc:237] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08e756b2ebb4cb3a2b3f172e274c8f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46639 } }
I20260810 00:56:38.165021 9964 sys_catalog.cc:455] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e08e756b2ebb4cb3a2b3f172e274c8f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08e756b2ebb4cb3a2b3f172e274c8f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46639 } } }
I20260810 00:56:38.165412 9964 sys_catalog.cc:458] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:38.165891 9965 sys_catalog.cc:455] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e08e756b2ebb4cb3a2b3f172e274c8f9. Latest consensus state: current_term: 1 leader_uuid: "e08e756b2ebb4cb3a2b3f172e274c8f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08e756b2ebb4cb3a2b3f172e274c8f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46639 } } }
I20260810 00:56:38.166162 9965 sys_catalog.cc:458] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:38.168300 9985 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:38.173661 9985 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:38.182505 9985 catalog_manager.cc:1347] Generated new cluster ID: 836e057077094524a8f2b4865d1f2938
I20260810 00:56:38.182597 9985 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:38.183173 9960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:38.165183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59183 (local address 127.0.0.1:46639)
0810 00:56:38.165356 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:38.165361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:38.167170 (+ 1809us) server_negotiation.cc:407] Connection header received
0810 00:56:38.167353 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:38.167358 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:38.167431 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:38.167537 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:38.169214 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:38.170355 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:38.172283 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:38.173008 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:38.173105 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:38.176286 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:38.176296 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:38.176301 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:38.176343 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:38.179750 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:38.180678 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:38.180684 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:38.180688 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:38.180834 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:38.181270 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:38.181275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:38.181278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:38.181661 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:38.181920 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:38.182478 (+ 558us) server_negotiation.cc:299] Negotiation successful
0810 00:56:38.182702 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 00:56:38.196393 9985 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:38.197510 9985 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:38.205595 9985 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e08e756b2ebb4cb3a2b3f172e274c8f9: Generated new TSK 0
I20260810 00:56:38.206249 9985 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:56:38.210920 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 9849
I20260810 00:56:38.211896 9948 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:38.297976 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 9849
2026-08-10T00:56:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2020 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:56:38 dist-test-slave-jlzn krb5kdc[10007](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:56:38 dist-test-slave-jlzn krb5kdc[10007](info): set up 2 sockets
Aug 10 00:56:38 dist-test-slave-jlzn krb5kdc[10007](info): commencing operation
krb5kdc: starting...
W20260810 00:56:39.487178 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:56:39 dist-test-slave-jlzn krb5kdc[10007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:56:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 00:56:39.736171 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43095
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:40.060597 10033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:40.060842 10033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:40.060964 10033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:40.067766 10033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:40.067883 10033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:40.067953 10033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:40.068033 10033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:40.068090 10033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:40.077251 10033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:40.078301 10033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:40.079433 10033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:40.090051 10039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:40.090015 10043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:40.089978 10040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:40.090893 10033 server_base.cc:1034] running on GCE node
I20260810 00:56:40.091557 10033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:40.092808 10033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:40.093997 10033 hybrid_clock.cc:648] HybridClock initialized: now 1786323400093955 us; error 35 us; skew 500 ppm
I20260810 00:56:40.094283 10033 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:40 dist-test-slave-jlzn krb5kdc[10007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323400, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:40.097150 10033 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 00:56:40.098416 10033 webserver.cc:460] Webserver started at http://127.0.0.1:41297/ using document root <none> and password file <none>
I20260810 00:56:40.098991 10033 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:40.099107 10033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:40.099380 10033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:40.102118 10033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "499252ccbc6e4f6cbe9b6f32e6545db8"
format_stamp: "Formatted at 2026-08-10 00:56:40 on dist-test-slave-jlzn"
server_key: "43932a17b60f9b32bb1da936acddfabf"
server_key_iv: "d066695ff8a86beaceb60291c8716ca3"
server_key_version: "encryptionkey@0"
I20260810 00:56:40.102828 10033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "499252ccbc6e4f6cbe9b6f32e6545db8"
format_stamp: "Formatted at 2026-08-10 00:56:40 on dist-test-slave-jlzn"
server_key: "43932a17b60f9b32bb1da936acddfabf"
server_key_iv: "d066695ff8a86beaceb60291c8716ca3"
server_key_version: "encryptionkey@0"
I20260810 00:56:40.107419 10033 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260810 00:56:40.110558 10060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:40.111716 10033 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:56:40.111881 10033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "499252ccbc6e4f6cbe9b6f32e6545db8"
format_stamp: "Formatted at 2026-08-10 00:56:40 on dist-test-slave-jlzn"
server_key: "43932a17b60f9b32bb1da936acddfabf"
server_key_iv: "d066695ff8a86beaceb60291c8716ca3"
server_key_version: "encryptionkey@0"
I20260810 00:56:40.112047 10033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:40.154569 10033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:40.158187 10033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:40.158445 10033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:40.175007 10033 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43095
I20260810 00:56:40.175037 10144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43095 every 8 connection(s)
I20260810 00:56:40.176250 10033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:40.179611 10146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:40.180927 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 10033
I20260810 00:56:40.181097 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:40.181572 10629 external_mini_cluster.cc:1428] Setting key 69b9003d9c25b1189137831c86f7d095
I20260810 00:56:40.193171 10146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8: Bootstrap starting.
I20260810 00:56:40.194979 10155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:40.183258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55875 (local address 127.0.0.1:43095)
0810 00:56:40.184088 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:40.184106 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:40.184172 (+ 66us) server_negotiation.cc:407] Connection header received
0810 00:56:40.185748 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:40.185779 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:40.186254 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:40.186675 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:40.187153 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:40.188931 (+ 1778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:40.190208 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:40.191247 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:40.191421 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:40.192437 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:40.192447 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:40.192462 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:40.193098 (+ 636us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:40.193212 (+ 114us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:40.193245 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:40.193353 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:40.193546 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 00:56:40.193785 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":338,"threads_started":1}
I20260810 00:56:40.196552 10146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:40.197645 10146 log.cc:824] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:40.198304 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:56:40.200500 10146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8: No bootstrap required, opened a new log
I20260810 00:56:40.208904 10155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:40.203150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53199 (local address 127.0.0.1:43095)
0810 00:56:40.203311 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:40.203315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:40.203555 (+ 240us) server_negotiation.cc:407] Connection header received
0810 00:56:40.203680 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:40.203685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:40.203755 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:40.203876 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:40.204329 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:40.205350 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:40.206547 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:40.207213 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:40.207330 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:40.208238 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:40.208245 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:40.208249 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:40.208269 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:40.208311 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:40.208330 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:40.208430 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:40.208530 (+ 100us) server_negotiation.cc:299] Negotiation successful
0810 00:56:40.208611 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 00:56:40.209645 10146 raft_consensus.cc:348] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499252ccbc6e4f6cbe9b6f32e6545db8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43095 } }
I20260810 00:56:40.209970 10146 raft_consensus.cc:374] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:40.210089 10146 raft_consensus.cc:729] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499252ccbc6e4f6cbe9b6f32e6545db8, State: Initialized, Role: FOLLOWER
I20260810 00:56:40.210618 10146 consensus_queue.cc:260] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499252ccbc6e4f6cbe9b6f32e6545db8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43095 } }
I20260810 00:56:40.210846 10146 raft_consensus.cc:388] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:40.210975 10146 raft_consensus.cc:482] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:40.211112 10146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:40.213129 10146 raft_consensus.cc:504] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499252ccbc6e4f6cbe9b6f32e6545db8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43095 } }
I20260810 00:56:40.213558 10146 leader_election.cc:302] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 499252ccbc6e4f6cbe9b6f32e6545db8; no voters:
I20260810 00:56:40.214061 10146 leader_election.cc:288] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:40.214181 10163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:40.215993 10146 sys_catalog.cc:564] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:40.220250 10163 raft_consensus.cc:686] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8 [term 1 LEADER]: Becoming Leader. State: Replica: 499252ccbc6e4f6cbe9b6f32e6545db8, State: Running, Role: LEADER
I20260810 00:56:40.220671 10163 consensus_queue.cc:237] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499252ccbc6e4f6cbe9b6f32e6545db8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43095 } }
I20260810 00:56:40.223342 10164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "499252ccbc6e4f6cbe9b6f32e6545db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499252ccbc6e4f6cbe9b6f32e6545db8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43095 } } }
I20260810 00:56:40.223690 10164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:40.223253 10165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 499252ccbc6e4f6cbe9b6f32e6545db8. Latest consensus state: current_term: 1 leader_uuid: "499252ccbc6e4f6cbe9b6f32e6545db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499252ccbc6e4f6cbe9b6f32e6545db8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43095 } } }
I20260810 00:56:40.224298 10165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8 [sys.catalog]: This master's current role is: LEADER
W20260810 00:56:40.227758 10181 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 00:56:40.227905 10181 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 00:56:40.228032 10184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:40.229611 10184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:40.237336 10184 catalog_manager.cc:1347] Generated new cluster ID: 9871029937824de1b959951d4a453898
I20260810 00:56:40.237475 10184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:40.244933 10184 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:40.246603 10184 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:40.252851 10184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 499252ccbc6e4f6cbe9b6f32e6545db8: Generated new TSK 0
I20260810 00:56:40.253712 10184 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:56:40.267187 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 10033
W20260810 00:56:40.268215 10138 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55875
W20260810 00:56:40.268880 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 10033: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:56:40.269007 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 10033
2026-08-10T00:56:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1972 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:56:40 dist-test-slave-jlzn krb5kdc[10195](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:56:40 dist-test-slave-jlzn krb5kdc[10195](info): set up 2 sockets
Aug 10 00:56:40 dist-test-slave-jlzn krb5kdc[10195](info): commencing operation
krb5kdc: starting...
W20260810 00:56:41.465167 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:56:41 dist-test-slave-jlzn krb5kdc[10195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:56:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 00:56:41.712916 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37193
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:42.026624 10213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:42.026886 10213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:42.027009 10213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:42.033876 10213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:42.033973 10213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:42.034050 10213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:42.034132 10213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:42.034191 10213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:42.043351 10213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:42.044415 10213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:42.045567 10213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:42.056397 10221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:42.056444 10223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:42.056473 10220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:42.057356 10213 server_base.cc:1034] running on GCE node
I20260810 00:56:42.058037 10213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:42.059206 10213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:42.060393 10213 hybrid_clock.cc:648] HybridClock initialized: now 1786323402060348 us; error 39 us; skew 500 ppm
I20260810 00:56:42.060698 10213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:42 dist-test-slave-jlzn krb5kdc[10195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323402, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:42.063540 10213 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 00:56:42.064818 10213 webserver.cc:460] Webserver started at http://127.0.0.1:39429/ using document root <none> and password file <none>
I20260810 00:56:42.065392 10213 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:42.065506 10213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:42.065790 10213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:42.068513 10213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "3bbf212a68bc461eaff63c0becea9cad"
format_stamp: "Formatted at 2026-08-10 00:56:42 on dist-test-slave-jlzn"
server_key: "c84c05e8de9761eb1bc137a29a7c036a"
server_key_iv: "267aac6c3447a3218edaa97a067c6959"
server_key_version: "encryptionkey@0"
I20260810 00:56:42.069236 10213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "3bbf212a68bc461eaff63c0becea9cad"
format_stamp: "Formatted at 2026-08-10 00:56:42 on dist-test-slave-jlzn"
server_key: "c84c05e8de9761eb1bc137a29a7c036a"
server_key_iv: "267aac6c3447a3218edaa97a067c6959"
server_key_version: "encryptionkey@0"
I20260810 00:56:42.073861 10213 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260810 00:56:42.076967 10237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:42.078110 10213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:56:42.078263 10213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "3bbf212a68bc461eaff63c0becea9cad"
format_stamp: "Formatted at 2026-08-10 00:56:42 on dist-test-slave-jlzn"
server_key: "c84c05e8de9761eb1bc137a29a7c036a"
server_key_iv: "267aac6c3447a3218edaa97a067c6959"
server_key_version: "encryptionkey@0"
I20260810 00:56:42.078429 10213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:42.083583 10213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:42.087009 10213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:42.087268 10213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:42.102963 10213 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37193
I20260810 00:56:42.103017 10305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37193 every 8 connection(s)
I20260810 00:56:42.104148 10213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:42.107443 10306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:42.113036 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 10213
I20260810 00:56:42.113209 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:42.113598 10629 external_mini_cluster.cc:1428] Setting key e2662fc2f4bd4bc131eb1d88b0562940
I20260810 00:56:42.117384 10306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad: Bootstrap starting.
I20260810 00:56:42.120417 10306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:42.120667 10309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:42.115167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40193 (local address 127.0.0.1:37193)
0810 00:56:42.115945 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:42.115956 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:42.115995 (+ 39us) server_negotiation.cc:407] Connection header received
0810 00:56:42.117556 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:42.117576 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:42.117869 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:42.118184 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:42.119152 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:42.119158 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:42.119171 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:42.119186 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:42.119241 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:42.119262 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:42.119457 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:42.119585 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 00:56:42.119777 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":309,"threads_started":1}
I20260810 00:56:42.121650 10306 log.cc:824] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:42.124253 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:56:42.124531 10306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad: No bootstrap required, opened a new log
I20260810 00:56:42.130618 10309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:42.128250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40085 (local address 127.0.0.1:37193)
0810 00:56:42.128432 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:42.128439 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:42.128933 (+ 494us) server_negotiation.cc:407] Connection header received
0810 00:56:42.129078 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:42.129083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:42.129128 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:42.129207 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:42.130016 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:42.130021 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:42.130026 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:42.130043 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:42.130084 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:42.130098 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:42.130206 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:42.130328 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 00:56:42.130382 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 00:56:42.132079 10306 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bbf212a68bc461eaff63c0becea9cad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37193 } }
I20260810 00:56:42.132395 10306 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:42.132480 10306 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bbf212a68bc461eaff63c0becea9cad, State: Initialized, Role: FOLLOWER
I20260810 00:56:42.132979 10306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bbf212a68bc461eaff63c0becea9cad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37193 } }
I20260810 00:56:42.133265 10306 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:42.133423 10306 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:42.133595 10306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:42.135635 10306 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bbf212a68bc461eaff63c0becea9cad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37193 } }
I20260810 00:56:42.136165 10306 leader_election.cc:302] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bbf212a68bc461eaff63c0becea9cad; no voters:
I20260810 00:56:42.136659 10306 leader_election.cc:288] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:42.136770 10321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:42.137393 10321 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad [term 1 LEADER]: Becoming Leader. State: Replica: 3bbf212a68bc461eaff63c0becea9cad, State: Running, Role: LEADER
I20260810 00:56:42.137907 10321 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bbf212a68bc461eaff63c0becea9cad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37193 } }
I20260810 00:56:42.138509 10306 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:42.140765 10325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bbf212a68bc461eaff63c0becea9cad. Latest consensus state: current_term: 1 leader_uuid: "3bbf212a68bc461eaff63c0becea9cad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bbf212a68bc461eaff63c0becea9cad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37193 } } }
I20260810 00:56:42.140846 10323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bbf212a68bc461eaff63c0becea9cad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bbf212a68bc461eaff63c0becea9cad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37193 } } }
I20260810 00:56:42.141135 10325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:42.141180 10323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:42.145501 10328 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:42.150913 10328 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:42.158952 10328 catalog_manager.cc:1347] Generated new cluster ID: c56d4ee976f64c4683e7b44bd33a4d68
I20260810 00:56:42.159060 10328 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:42.178141 10328 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:42.178565 10328 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:42.184329 10328 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bbf212a68bc461eaff63c0becea9cad: Generated new TSK 0
I20260810 00:56:42.184971 10328 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:56:42.193774 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 10213
W20260810 00:56:42.194697 10300 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40193
W20260810 00:56:42.195278 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 10213: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:56:42.195426 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 10213
2026-08-10T00:56:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1926 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:56:42 dist-test-slave-jlzn krb5kdc[10353](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:56:42 dist-test-slave-jlzn krb5kdc[10353](info): set up 2 sockets
Aug 10 00:56:42 dist-test-slave-jlzn krb5kdc[10353](info): commencing operation
krb5kdc: starting...
W20260810 00:56:43.386873 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.002s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:56:43 dist-test-slave-jlzn krb5kdc[10353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:56:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 00:56:43.632467 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43783
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:43.949883 10372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:43.950151 10372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:43.950250 10372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:43.957145 10372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:43.957242 10372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:43.957295 10372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:43.957355 10372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:43.957396 10372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:43.966751 10372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:43.967831 10372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:43.969017 10372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:43.979638 10379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:43.979681 10381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:43.979704 10385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:43.980719 10372 server_base.cc:1034] running on GCE node
I20260810 00:56:43.981431 10372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:43.982620 10372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:43.983860 10372 hybrid_clock.cc:648] HybridClock initialized: now 1786323403983792 us; error 25 us; skew 500 ppm
I20260810 00:56:43.984270 10372 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:43 dist-test-slave-jlzn krb5kdc[10353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323403, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:43.990293 10372 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 00:56:43.991590 10372 webserver.cc:460] Webserver started at http://127.0.0.1:41963/ using document root <none> and password file <none>
I20260810 00:56:43.992231 10372 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:43.992332 10372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:43.992563 10372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:43.995262 10372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "781589325b774a79af35db5c01d9e430"
format_stamp: "Formatted at 2026-08-10 00:56:43 on dist-test-slave-jlzn"
server_key: "057a3efd4589efe8ae5d12c759eddd71"
server_key_iv: "5b9bd47c61e94e1ae619588bbbffe7d1"
server_key_version: "encryptionkey@0"
I20260810 00:56:43.995998 10372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "781589325b774a79af35db5c01d9e430"
format_stamp: "Formatted at 2026-08-10 00:56:43 on dist-test-slave-jlzn"
server_key: "057a3efd4589efe8ae5d12c759eddd71"
server_key_iv: "5b9bd47c61e94e1ae619588bbbffe7d1"
server_key_version: "encryptionkey@0"
I20260810 00:56:44.000568 10372 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 00:56:44.003614 10394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:44.004814 10372 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:56:44.004958 10372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "781589325b774a79af35db5c01d9e430"
format_stamp: "Formatted at 2026-08-10 00:56:43 on dist-test-slave-jlzn"
server_key: "057a3efd4589efe8ae5d12c759eddd71"
server_key_iv: "5b9bd47c61e94e1ae619588bbbffe7d1"
server_key_version: "encryptionkey@0"
I20260810 00:56:44.005119 10372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:44.029989 10372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:44.033506 10372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:44.033754 10372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:44.050055 10372 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43783
I20260810 00:56:44.050100 10463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43783 every 8 connection(s)
I20260810 00:56:44.051230 10372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:44.053068 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 10372
I20260810 00:56:44.053229 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:44.053648 10629 external_mini_cluster.cc:1428] Setting key 2f5014d76fa3c5c2847738ed73c7f75b
I20260810 00:56:44.054941 10464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 00:56:44 dist-test-slave-jlzn krb5kdc[10353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 00:56:44.067904 10464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430: Bootstrap starting.
I20260810 00:56:44.070858 10464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:44.072005 10464 log.cc:824] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:44.074818 10464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430: No bootstrap required, opened a new log
I20260810 00:56:44.075338 10467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:44.055305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:55881 (local address 127.0.0.1:43783)
0810 00:56:44.056031 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:44.056042 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:44.056081 (+ 39us) server_negotiation.cc:407] Connection header received
0810 00:56:44.057604 (+ 1523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:44.057625 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:44.057970 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:44.058290 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:44.060071 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:44.061385 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:44.063088 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:44.063699 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:44.063850 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:44.068491 (+ 4641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:44.068498 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:44.068509 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:44.068547 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:44.071324 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:44.072193 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:44.072199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:44.072205 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:44.072359 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:44.072860 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:44.072865 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:44.072868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:44.073421 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:44.073663 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:44.074134 (+ 471us) server_negotiation.cc:299] Negotiation successful
0810 00:56:44.074381 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":311,"threads_started":1}
I20260810 00:56:44.078994 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:56:44.084293 10464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "781589325b774a79af35db5c01d9e430" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43783 } }
I20260810 00:56:44.084609 10464 raft_consensus.cc:374] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:44.084697 10464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 781589325b774a79af35db5c01d9e430, State: Initialized, Role: FOLLOWER
I20260810 00:56:44.085222 10464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "781589325b774a79af35db5c01d9e430" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43783 } }
I20260810 00:56:44.085440 10464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:44.085561 10464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:44.085706 10464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:44.087755 10464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "781589325b774a79af35db5c01d9e430" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43783 } }
I20260810 00:56:44.088231 10464 leader_election.cc:302] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 781589325b774a79af35db5c01d9e430; no voters:
I20260810 00:56:44.088815 10464 leader_election.cc:288] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:44.088922 10478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:44.089579 10478 raft_consensus.cc:686] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430 [term 1 LEADER]: Becoming Leader. State: Replica: 781589325b774a79af35db5c01d9e430, State: Running, Role: LEADER
I20260810 00:56:44.090194 10478 consensus_queue.cc:237] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "781589325b774a79af35db5c01d9e430" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43783 } }
I20260810 00:56:44.092478 10479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 781589325b774a79af35db5c01d9e430. Latest consensus state: current_term: 1 leader_uuid: "781589325b774a79af35db5c01d9e430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "781589325b774a79af35db5c01d9e430" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43783 } } }
I20260810 00:56:44.092875 10479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:44.093379 10464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:44.095500 10480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "781589325b774a79af35db5c01d9e430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "781589325b774a79af35db5c01d9e430" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43783 } } }
I20260810 00:56:44.095824 10480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:44.099296 10467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:44.084085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:42991 (local address 127.0.0.1:43783)
0810 00:56:44.084248 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:44.084253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:44.084626 (+ 373us) server_negotiation.cc:407] Connection header received
0810 00:56:44.084805 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:44.084809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:44.084862 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:44.084935 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:44.086321 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:44.087090 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:44.088395 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:44.088878 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:44.088998 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:44.091458 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:44.091468 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:44.091472 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:44.091517 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:44.095223 (+ 3706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:44.096263 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:44.096268 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:44.096272 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:44.096426 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:44.097022 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:44.097027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:44.097030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:44.097397 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:44.098548 (+ 1151us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:44.098730 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 00:56:44.098901 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 00:56:44.100911 10485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:44.105373 10485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:44.113097 10485 catalog_manager.cc:1347] Generated new cluster ID: e030db1c0d71497bb4afdb6e40583722
I20260810 00:56:44.113200 10485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:44.134533 10485 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:44.136232 10485 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:44.144366 10485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 781589325b774a79af35db5c01d9e430: Generated new TSK 0
I20260810 00:56:44.145219 10485 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:56:44.154017 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 10372
I20260810 00:56:44.154939 10458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:44.243183 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 10372
W20260810 00:56:44.264156 10351 connection.cc:535] client connection to 127.0.0.1:43783 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43783): Connection reset by peer (error 104)
2026-08-10T00:56:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2045 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:56:44 dist-test-slave-jlzn krb5kdc[10519](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:56:44 dist-test-slave-jlzn krb5kdc[10519](info): set up 2 sockets
Aug 10 00:56:44 dist-test-slave-jlzn krb5kdc[10519](info): commencing operation
krb5kdc: starting...
W20260810 00:56:45.453822 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:56:45 dist-test-slave-jlzn krb5kdc[10519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:56:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 00:56:45.705926 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34675
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:46.025200 10542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:46.025463 10542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:46.025571 10542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:46.032482 10542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:46.032586 10542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:46.032639 10542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:46.032697 10542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:46.032740 10542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:46.041914 10542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:46.043004 10542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:46.044212 10542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:46.054721 10551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:46.054838 10552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:46.054863 10554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:46.055756 10542 server_base.cc:1034] running on GCE node
I20260810 00:56:46.056468 10542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:46.057682 10542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:46.058876 10542 hybrid_clock.cc:648] HybridClock initialized: now 1786323406058835 us; error 34 us; skew 500 ppm
I20260810 00:56:46.059176 10542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:46 dist-test-slave-jlzn krb5kdc[10519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323406, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:46.062150 10542 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 00:56:46.063421 10542 webserver.cc:460] Webserver started at http://127.0.0.1:46435/ using document root <none> and password file <none>
I20260810 00:56:46.064049 10542 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:46.064164 10542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:46.064431 10542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:46.067108 10542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "b87ea58770184f84b433383cc0c154ee"
format_stamp: "Formatted at 2026-08-10 00:56:46 on dist-test-slave-jlzn"
server_key: "9c9c8182fe5f53e3942088848c7226fb"
server_key_iv: "2c4e4c6fbc82cb1487650af2277efd1c"
server_key_version: "encryptionkey@0"
I20260810 00:56:46.067832 10542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "b87ea58770184f84b433383cc0c154ee"
format_stamp: "Formatted at 2026-08-10 00:56:46 on dist-test-slave-jlzn"
server_key: "9c9c8182fe5f53e3942088848c7226fb"
server_key_iv: "2c4e4c6fbc82cb1487650af2277efd1c"
server_key_version: "encryptionkey@0"
I20260810 00:56:46.072414 10542 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:56:46.075518 10565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:46.076709 10542 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:56:46.076859 10542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "b87ea58770184f84b433383cc0c154ee"
format_stamp: "Formatted at 2026-08-10 00:56:46 on dist-test-slave-jlzn"
server_key: "9c9c8182fe5f53e3942088848c7226fb"
server_key_iv: "2c4e4c6fbc82cb1487650af2277efd1c"
server_key_version: "encryptionkey@0"
I20260810 00:56:46.077023 10542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:46.095754 10542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:46.099464 10542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:46.099713 10542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:46.116192 10542 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34675
I20260810 00:56:46.116243 10651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34675 every 8 connection(s)
I20260810 00:56:46.117465 10542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:46.120750 10652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:46.126312 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 10542
I20260810 00:56:46.126499 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:46.127039 10629 external_mini_cluster.cc:1428] Setting key b6b6aba8d47579c9be0aa2aea6580cd1
I20260810 00:56:46.130980 10652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee: Bootstrap starting.
I20260810 00:56:46.134078 10652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:46.135241 10652 log.cc:824] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:46.138224 10652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee: No bootstrap required, opened a new log
I20260810 00:56:46.139849 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:46.128791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:53813 (local address 127.0.0.1:34675)
0810 00:56:46.129513 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:46.129523 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:46.129563 (+ 40us) server_negotiation.cc:407] Connection header received
0810 00:56:46.131164 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:46.131185 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:46.131524 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:46.131839 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:46.132407 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:46.133659 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:46.135430 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:46.136436 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:46.136592 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:46.137759 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:46.137767 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:46.137779 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:46.138244 (+ 465us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:46.138322 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:46.138358 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:46.138466 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:46.138747 (+ 281us) server_negotiation.cc:299] Negotiation successful
0810 00:56:46.138905 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":335,"threads_started":1}
I20260810 00:56:46.143237 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:56:46.145634 10652 raft_consensus.cc:348] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87ea58770184f84b433383cc0c154ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34675 } }
I20260810 00:56:46.146035 10652 raft_consensus.cc:374] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:46.146145 10652 raft_consensus.cc:729] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b87ea58770184f84b433383cc0c154ee, State: Initialized, Role: FOLLOWER
I20260810 00:56:46.146674 10652 consensus_queue.cc:260] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87ea58770184f84b433383cc0c154ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34675 } }
I20260810 00:56:46.146889 10652 raft_consensus.cc:388] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:46.147027 10652 raft_consensus.cc:482] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:46.147239 10652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:46.149886 10652 raft_consensus.cc:504] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87ea58770184f84b433383cc0c154ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34675 } }
I20260810 00:56:46.150494 10652 leader_election.cc:302] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b87ea58770184f84b433383cc0c154ee; no voters:
I20260810 00:56:46.151099 10652 leader_election.cc:288] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:46.151193 10664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:46.152704 10664 raft_consensus.cc:686] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee [term 1 LEADER]: Becoming Leader. State: Replica: b87ea58770184f84b433383cc0c154ee, State: Running, Role: LEADER
I20260810 00:56:46.153398 10652 sys_catalog.cc:564] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:46.153527 10664 consensus_queue.cc:237] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87ea58770184f84b433383cc0c154ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34675 } }
I20260810 00:56:46.154366 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:46.147278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:42213 (local address 127.0.0.1:34675)
0810 00:56:46.147436 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:46.147442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:46.147917 (+ 475us) server_negotiation.cc:407] Connection header received
0810 00:56:46.148110 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:46.148113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:46.148172 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:46.148243 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:46.148817 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:46.149527 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:46.151338 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:46.152136 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:46.152271 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:46.153389 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:46.153397 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:46.153402 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:46.153422 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:46.153484 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:46.153509 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:46.153625 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:46.154012 (+ 387us) server_negotiation.cc:299] Negotiation successful
0810 00:56:46.154062 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 00:56:46.156213 10666 sys_catalog.cc:455] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader b87ea58770184f84b433383cc0c154ee. Latest consensus state: current_term: 1 leader_uuid: "b87ea58770184f84b433383cc0c154ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87ea58770184f84b433383cc0c154ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34675 } } }
I20260810 00:56:46.156507 10666 sys_catalog.cc:458] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:46.157001 10665 sys_catalog.cc:455] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b87ea58770184f84b433383cc0c154ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87ea58770184f84b433383cc0c154ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34675 } } }
I20260810 00:56:46.157320 10665 sys_catalog.cc:458] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:46.158790 10670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:46.163615 10670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:46.173162 10670 catalog_manager.cc:1347] Generated new cluster ID: 4a7b2fafecfd404bb95edcf57c335be7
I20260810 00:56:46.173249 10670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:46.188832 10670 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:46.189988 10670 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:46.197719 10670 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b87ea58770184f84b433383cc0c154ee: Generated new TSK 0
I20260810 00:56:46.198581 10670 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:56:46.212119 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 10542
W20260810 00:56:46.213585 10641 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.68:53813
W20260810 00:56:46.214308 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 10542: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:56:46.214449 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 10542
W20260810 00:56:46.239876 10517 connection.cc:535] client connection to 127.0.0.1:34675 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34675): Connection reset by peer (error 104)
2026-08-10T00:56:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1975 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:56:46 dist-test-slave-jlzn krb5kdc[10695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:56:46 dist-test-slave-jlzn krb5kdc[10695](info): set up 2 sockets
Aug 10 00:56:46 dist-test-slave-jlzn krb5kdc[10695](info): commencing operation
krb5kdc: starting...
W20260810 00:56:47.421681 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.001s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:56:47 dist-test-slave-jlzn krb5kdc[10695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:56:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 00:56:47.673115 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34283
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:47.992650 10721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:47.992898 10721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:47.992995 10721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:47.999819 10721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:47.999924 10721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:47.999980 10721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:48.000053 10721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:48.000098 10721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:48.009217 10721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:48.010254 10721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:48.011425 10721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:48.022558 10737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:48.022495 10733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:48.022563 10730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:48.023361 10721 server_base.cc:1034] running on GCE node
I20260810 00:56:48.024142 10721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:48.025372 10721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:48.026577 10721 hybrid_clock.cc:648] HybridClock initialized: now 1786323408026534 us; error 38 us; skew 500 ppm
I20260810 00:56:48.026890 10721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:48 dist-test-slave-jlzn krb5kdc[10695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323408, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:48.029856 10721 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 00:56:48.031153 10721 webserver.cc:460] Webserver started at http://127.0.0.1:32979/ using document root <none> and password file <none>
I20260810 00:56:48.031772 10721 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:48.031929 10721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:48.032189 10721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:48.034889 10721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "e12995db250f4064834a2122cb6d54c2"
format_stamp: "Formatted at 2026-08-10 00:56:48 on dist-test-slave-jlzn"
server_key: "18d428bed592a3e3610ebdeefaef182d"
server_key_iv: "6b7f3c6c80a0bcfcae1061fa83af9db1"
server_key_version: "encryptionkey@0"
I20260810 00:56:48.035588 10721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "e12995db250f4064834a2122cb6d54c2"
format_stamp: "Formatted at 2026-08-10 00:56:48 on dist-test-slave-jlzn"
server_key: "18d428bed592a3e3610ebdeefaef182d"
server_key_iv: "6b7f3c6c80a0bcfcae1061fa83af9db1"
server_key_version: "encryptionkey@0"
I20260810 00:56:48.040293 10721 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 00:56:48.043385 10747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:48.044625 10721 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:56:48.044796 10721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "e12995db250f4064834a2122cb6d54c2"
format_stamp: "Formatted at 2026-08-10 00:56:48 on dist-test-slave-jlzn"
server_key: "18d428bed592a3e3610ebdeefaef182d"
server_key_iv: "6b7f3c6c80a0bcfcae1061fa83af9db1"
server_key_version: "encryptionkey@0"
I20260810 00:56:48.044971 10721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:48.050307 10721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:48.054221 10721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:48.054487 10721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:48.071169 10721 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34283
I20260810 00:56:48.071223 10823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34283 every 8 connection(s)
I20260810 00:56:48.072402 10721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:48.073249 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 10721
I20260810 00:56:48.073441 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:48.073946 10629 external_mini_cluster.cc:1428] Setting key 32fe0294ffb889c94b2497c4d0c53207
I20260810 00:56:48.076443 10824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:48.080906 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:48.075592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:37779 (local address 127.0.0.1:34283)
0810 00:56:48.076300 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:48.076310 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:48.076349 (+ 39us) server_negotiation.cc:407] Connection header received
0810 00:56:48.077867 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:48.077887 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:48.078176 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:48.078467 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:48.079411 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:48.079417 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:48.079429 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:48.079442 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:48.079532 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:48.079557 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:48.079734 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:48.079880 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 00:56:48.080054 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":308,"threads_started":1}
I20260810 00:56:48.084086 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:56:48.088045 10824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2: Bootstrap starting.
I20260810 00:56:48.091290 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:48.089093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:37971 (local address 127.0.0.1:34283)
0810 00:56:48.089255 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:48.089261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:48.089599 (+ 338us) server_negotiation.cc:407] Connection header received
0810 00:56:48.089754 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:48.089757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:48.089799 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:48.089873 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:48.090715 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:48.090720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:48.090722 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:48.090739 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:48.090772 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:48.090786 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:48.090866 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:48.091005 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 00:56:48.091047 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 00:56:48.091606 10824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:48.092727 10824 log.cc:824] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:48.095036 10824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2: No bootstrap required, opened a new log
I20260810 00:56:48.101847 10824 raft_consensus.cc:348] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e12995db250f4064834a2122cb6d54c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34283 } }
I20260810 00:56:48.102155 10824 raft_consensus.cc:374] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:48.102252 10824 raft_consensus.cc:729] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e12995db250f4064834a2122cb6d54c2, State: Initialized, Role: FOLLOWER
I20260810 00:56:48.102682 10824 consensus_queue.cc:260] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e12995db250f4064834a2122cb6d54c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34283 } }
I20260810 00:56:48.102865 10824 raft_consensus.cc:388] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:48.102958 10824 raft_consensus.cc:482] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:48.103076 10824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:48.105257 10824 raft_consensus.cc:504] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e12995db250f4064834a2122cb6d54c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34283 } }
I20260810 00:56:48.105670 10824 leader_election.cc:302] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e12995db250f4064834a2122cb6d54c2; no voters:
I20260810 00:56:48.106181 10824 leader_election.cc:288] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:48.106277 10839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:48.106894 10839 raft_consensus.cc:686] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2 [term 1 LEADER]: Becoming Leader. State: Replica: e12995db250f4064834a2122cb6d54c2, State: Running, Role: LEADER
I20260810 00:56:48.107353 10839 consensus_queue.cc:237] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e12995db250f4064834a2122cb6d54c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34283 } }
I20260810 00:56:48.108013 10824 sys_catalog.cc:564] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:48.109812 10842 sys_catalog.cc:455] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e12995db250f4064834a2122cb6d54c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e12995db250f4064834a2122cb6d54c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34283 } } }
I20260810 00:56:48.110198 10842 sys_catalog.cc:458] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:48.113925 10843 sys_catalog.cc:455] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e12995db250f4064834a2122cb6d54c2. Latest consensus state: current_term: 1 leader_uuid: "e12995db250f4064834a2122cb6d54c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e12995db250f4064834a2122cb6d54c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34283 } } }
I20260810 00:56:48.114156 10843 sys_catalog.cc:458] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:48.115087 10849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:48.119096 10849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:48.127188 10849 catalog_manager.cc:1347] Generated new cluster ID: 986d2eac021145248f88d329e77708d3
I20260810 00:56:48.127282 10849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:48.162078 10849 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:48.162462 10849 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:48.179621 10849 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e12995db250f4064834a2122cb6d54c2: Generated new TSK 0
I20260810 00:56:48.180325 10849 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:56:48.186772 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 10721
W20260810 00:56:48.188028 10817 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.68:37779
W20260810 00:56:48.188637 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 10721: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:56:48.188783 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 10721
2026-08-10T00:56:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1971 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:56:48 dist-test-slave-jlzn krb5kdc[10871](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:56:48 dist-test-slave-jlzn krb5kdc[10871](info): set up 2 sockets
Aug 10 00:56:48 dist-test-slave-jlzn krb5kdc[10871](info): commencing operation
krb5kdc: starting...
W20260810 00:56:49.385408 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:56:49 dist-test-slave-jlzn krb5kdc[10871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:56:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 00:56:49.638118 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39993
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:49.957616 10884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:49.957860 10884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:49.957950 10884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:49.964726 10884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:49.964820 10884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:49.964871 10884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:49.964929 10884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:49.964972 10884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:49.974005 10884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39993
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:49.975045 10884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:49.976208 10884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:49.986824 10897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:49.986881 10893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:49.986873 10895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:49.987619 10884 server_base.cc:1034] running on GCE node
I20260810 00:56:49.988360 10884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:49.989558 10884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:49.990762 10884 hybrid_clock.cc:648] HybridClock initialized: now 1786323409990715 us; error 38 us; skew 500 ppm
I20260810 00:56:49.991076 10884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:49 dist-test-slave-jlzn krb5kdc[10871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323409, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:49.996984 10884 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 00:56:49.998428 10884 webserver.cc:460] Webserver started at http://127.0.0.1:45747/ using document root <none> and password file <none>
I20260810 00:56:49.998991 10884 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:49.999099 10884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:49.999330 10884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:50.002039 10884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "dbcf5cc7c9c1455a9fdb8362866132ba"
format_stamp: "Formatted at 2026-08-10 00:56:49 on dist-test-slave-jlzn"
server_key: "5e1e3c8656a0e15edb4e69d037fd6cee"
server_key_iv: "e644652456c3cd95cac5efbb78f946ff"
server_key_version: "encryptionkey@0"
I20260810 00:56:50.002713 10884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "dbcf5cc7c9c1455a9fdb8362866132ba"
format_stamp: "Formatted at 2026-08-10 00:56:49 on dist-test-slave-jlzn"
server_key: "5e1e3c8656a0e15edb4e69d037fd6cee"
server_key_iv: "e644652456c3cd95cac5efbb78f946ff"
server_key_version: "encryptionkey@0"
I20260810 00:56:50.007266 10884 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 00:56:50.010401 10909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:50.011574 10884 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:56:50.011730 10884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "dbcf5cc7c9c1455a9fdb8362866132ba"
format_stamp: "Formatted at 2026-08-10 00:56:49 on dist-test-slave-jlzn"
server_key: "5e1e3c8656a0e15edb4e69d037fd6cee"
server_key_iv: "e644652456c3cd95cac5efbb78f946ff"
server_key_version: "encryptionkey@0"
I20260810 00:56:50.011935 10884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:50.025959 10884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:50.029413 10884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:50.029649 10884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:50.045807 10884 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39993
I20260810 00:56:50.045856 10984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39993 every 8 connection(s)
I20260810 00:56:50.046952 10884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:50.048343 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 10884
I20260810 00:56:50.048501 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:50.048895 10629 external_mini_cluster.cc:1428] Setting key 743416ac7c8acb74f16443fa1dd746c4
I20260810 00:56:50.050711 10985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 00:56:50 dist-test-slave-jlzn krb5kdc[10871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 00:56:50.062793 10985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba: Bootstrap starting.
I20260810 00:56:50.066327 10985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:50.067569 10985 log.cc:824] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:50.070631 10985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba: No bootstrap required, opened a new log
I20260810 00:56:50.071658 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:50.050442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:33707 (local address 127.0.0.1:39993)
0810 00:56:50.051219 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:50.051233 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:50.051277 (+ 44us) server_negotiation.cc:407] Connection header received
0810 00:56:50.053391 (+ 2114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:50.053416 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:50.053875 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:50.054271 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:50.055570 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:50.057537 (+ 1967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:50.058739 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:50.059723 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:50.059939 (+ 216us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:50.063699 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:50.063710 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:50.063726 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:50.063766 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:50.067399 (+ 3633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:50.068192 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:50.068200 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:50.068208 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:50.068374 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:50.068889 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:50.068893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:50.068897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:50.069500 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:50.069810 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:50.070096 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 00:56:50.070396 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":317,"threads_started":1}
I20260810 00:56:50.075109 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:56:50.078271 10985 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbcf5cc7c9c1455a9fdb8362866132ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260810 00:56:50.078565 10985 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:50.078657 10985 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcf5cc7c9c1455a9fdb8362866132ba, State: Initialized, Role: FOLLOWER
I20260810 00:56:50.079262 10985 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbcf5cc7c9c1455a9fdb8362866132ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260810 00:56:50.079486 10985 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:50.079617 10985 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:50.079773 10985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:50.082201 10985 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbcf5cc7c9c1455a9fdb8362866132ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260810 00:56:50.082713 10985 leader_election.cc:302] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbcf5cc7c9c1455a9fdb8362866132ba; no voters:
I20260810 00:56:50.083279 10985 leader_election.cc:288] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:50.083376 11000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:50.085381 10985 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:50.089960 11000 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba [term 1 LEADER]: Becoming Leader. State: Replica: dbcf5cc7c9c1455a9fdb8362866132ba, State: Running, Role: LEADER
I20260810 00:56:50.090505 11000 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbcf5cc7c9c1455a9fdb8362866132ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260810 00:56:50.093091 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:50.079183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:36339 (local address 127.0.0.1:39993)
0810 00:56:50.079348 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:50.079353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:50.079933 (+ 580us) server_negotiation.cc:407] Connection header received
0810 00:56:50.080098 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:50.080103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:50.080174 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:50.080274 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:50.081454 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:50.082495 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:50.083866 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:50.084500 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:50.084570 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:50.087003 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:50.087008 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:50.087011 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:50.087038 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:50.089791 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:50.090504 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:50.090510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:50.090514 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:50.090656 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:50.091085 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:50.091089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:50.091093 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:50.091363 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:50.091667 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:50.092506 (+ 839us) server_negotiation.cc:299] Negotiation successful
0810 00:56:50.092701 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 00:56:50.093835 11002 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbcf5cc7c9c1455a9fdb8362866132ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbcf5cc7c9c1455a9fdb8362866132ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } } }
I20260810 00:56:50.094282 11002 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:50.094419 11005 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbcf5cc7c9c1455a9fdb8362866132ba. Latest consensus state: current_term: 1 leader_uuid: "dbcf5cc7c9c1455a9fdb8362866132ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbcf5cc7c9c1455a9fdb8362866132ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } } }
I20260810 00:56:50.094780 11005 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:50.096000 11018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:50.101609 11018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:50.109935 11018 catalog_manager.cc:1347] Generated new cluster ID: 0339f5b5fc5449cc86c0eca0d1d6add8
I20260810 00:56:50.110031 11018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:50.142032 11018 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:50.143383 11018 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:50.154204 11018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbcf5cc7c9c1455a9fdb8362866132ba: Generated new TSK 0
I20260810 00:56:50.154874 11018 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:56:50.160020 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 10884
I20260810 00:56:50.161480 10979 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:50.250047 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 10884
W20260810 00:56:50.273591 10867 connection.cc:535] client connection to 127.0.0.1:39993 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39993): Connection reset by peer (error 104)
2026-08-10T00:56:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2062 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:56:50 dist-test-slave-jlzn krb5kdc[11036](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:56:50 dist-test-slave-jlzn krb5kdc[11036](info): set up 2 sockets
Aug 10 00:56:50 dist-test-slave-jlzn krb5kdc[11036](info): commencing operation
krb5kdc: starting...
W20260810 00:56:51.452224 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:56:51 dist-test-slave-jlzn krb5kdc[11036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:56:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 00:56:51.704715 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35781
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:52.023343 11054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:52.023613 11054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:52.023761 11054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:52.030544 11054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:52.030654 11054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:52.030735 11054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:52.030821 11054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:52.030884 11054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:52.040097 11054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35781
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:52.041180 11054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:52.042339 11054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:52.053156 11059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:52.053188 11062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:52.053109 11060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:52.054066 11054 server_base.cc:1034] running on GCE node
I20260810 00:56:52.054766 11054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:52.055976 11054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:52.057181 11054 hybrid_clock.cc:648] HybridClock initialized: now 1786323412057130 us; error 35 us; skew 500 ppm
I20260810 00:56:52.057490 11054 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:52 dist-test-slave-jlzn krb5kdc[11036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323412, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:52.060395 11054 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 00:56:52.061648 11054 webserver.cc:460] Webserver started at http://127.0.0.1:34811/ using document root <none> and password file <none>
I20260810 00:56:52.062249 11054 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:52.062366 11054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:52.062635 11054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:52.065366 11054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "967af7d521f5440e9cbf1c735aa78cf4"
format_stamp: "Formatted at 2026-08-10 00:56:52 on dist-test-slave-jlzn"
server_key: "520da3645c9c36da1ea235b82d58c86d"
server_key_iv: "e051ddee11e6781ead4b9530a3f11c22"
server_key_version: "encryptionkey@0"
I20260810 00:56:52.066048 11054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "967af7d521f5440e9cbf1c735aa78cf4"
format_stamp: "Formatted at 2026-08-10 00:56:52 on dist-test-slave-jlzn"
server_key: "520da3645c9c36da1ea235b82d58c86d"
server_key_iv: "e051ddee11e6781ead4b9530a3f11c22"
server_key_version: "encryptionkey@0"
I20260810 00:56:52.070683 11054 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 00:56:52.073750 11072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:52.074923 11054 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 00:56:52.075083 11054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "967af7d521f5440e9cbf1c735aa78cf4"
format_stamp: "Formatted at 2026-08-10 00:56:52 on dist-test-slave-jlzn"
server_key: "520da3645c9c36da1ea235b82d58c86d"
server_key_iv: "e051ddee11e6781ead4b9530a3f11c22"
server_key_version: "encryptionkey@0"
I20260810 00:56:52.075258 11054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:52.108104 11054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:52.112004 11054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:52.112267 11054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:52.128764 11054 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35781
I20260810 00:56:52.128805 11150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35781 every 8 connection(s)
I20260810 00:56:52.129968 11054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:52.133322 11151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:52.135423 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 11054
I20260810 00:56:52.135576 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:52.136059 10629 external_mini_cluster.cc:1428] Setting key 7827894e76b61cf034881f920772e247
I20260810 00:56:52.145761 11151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4: Bootstrap starting.
I20260810 00:56:52.149353 11151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:52.150364 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:52.137717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:35477 (local address 127.0.0.1:35781)
0810 00:56:52.138521 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:52.138535 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:52.138581 (+ 46us) server_negotiation.cc:407] Connection header received
0810 00:56:52.140877 (+ 2296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:52.140903 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:52.141358 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:52.141757 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:52.142252 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:52.144229 (+ 1977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:52.145552 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:52.146524 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:52.146700 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:52.147826 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:52.147836 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:52.147852 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:52.148487 (+ 635us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:52.148578 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:52.148617 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:52.148764 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:52.148967 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 00:56:52.149123 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":331,"threads_started":1}
I20260810 00:56:52.150669 11151 log.cc:824] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:52.153457 11151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4: No bootstrap required, opened a new log
I20260810 00:56:52.153998 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:56:52.161309 11151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967af7d521f5440e9cbf1c735aa78cf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35781 } }
I20260810 00:56:52.161620 11151 raft_consensus.cc:374] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:52.161739 11151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 967af7d521f5440e9cbf1c735aa78cf4, State: Initialized, Role: FOLLOWER
I20260810 00:56:52.162364 11151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967af7d521f5440e9cbf1c735aa78cf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35781 } }
I20260810 00:56:52.162675 11151 raft_consensus.cc:388] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:52.162818 11151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:52.162958 11151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:52.164291 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:52.158122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:55933 (local address 127.0.0.1:35781)
0810 00:56:52.158308 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:52.158314 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:52.158670 (+ 356us) server_negotiation.cc:407] Connection header received
0810 00:56:52.158780 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:52.158785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:52.158850 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:52.158954 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:52.159410 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:52.160548 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:52.161754 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:52.162393 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:52.162517 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:52.163523 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:52.163530 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:52.163537 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:52.163558 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:52.163624 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:52.163645 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:52.163761 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:52.163923 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 00:56:52.163975 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 00:56:52.165534 11151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967af7d521f5440e9cbf1c735aa78cf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35781 } }
I20260810 00:56:52.166019 11151 leader_election.cc:302] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 967af7d521f5440e9cbf1c735aa78cf4; no voters:
I20260810 00:56:52.166510 11151 leader_election.cc:288] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:52.166617 11166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:52.167286 11166 raft_consensus.cc:686] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4 [term 1 LEADER]: Becoming Leader. State: Replica: 967af7d521f5440e9cbf1c735aa78cf4, State: Running, Role: LEADER
I20260810 00:56:52.167757 11166 consensus_queue.cc:237] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967af7d521f5440e9cbf1c735aa78cf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35781 } }
I20260810 00:56:52.168378 11151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:52.170620 11168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "967af7d521f5440e9cbf1c735aa78cf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967af7d521f5440e9cbf1c735aa78cf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35781 } } }
I20260810 00:56:52.171060 11168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:52.170608 11170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 967af7d521f5440e9cbf1c735aa78cf4. Latest consensus state: current_term: 1 leader_uuid: "967af7d521f5440e9cbf1c735aa78cf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967af7d521f5440e9cbf1c735aa78cf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35781 } } }
I20260810 00:56:52.174724 11170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:52.174854 11179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:52.179409 11179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:52.188338 11179 catalog_manager.cc:1347] Generated new cluster ID: 532e1c5856cf4ef790af83ee567780e7
I20260810 00:56:52.188434 11179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:52.221978 11179 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:52.223165 11179 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:52.236279 11179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 967af7d521f5440e9cbf1c735aa78cf4: Generated new TSK 0
I20260810 00:56:52.236999 11179 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:56:52.246125 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 11054
W20260810 00:56:52.247320 11144 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.68:35477
W20260810 00:56:52.248128 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 11054: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:56:52.248272 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 11054
W20260810 00:56:52.272413 11034 connection.cc:535] client connection to 127.0.0.1:35781 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35781): Connection reset by peer (error 104)
2026-08-10T00:56:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1998 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:56:52 dist-test-slave-jlzn krb5kdc[11201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:56:52 dist-test-slave-jlzn krb5kdc[11201](info): set up 2 sockets
Aug 10 00:56:52 dist-test-slave-jlzn krb5kdc[11201](info): commencing operation
krb5kdc: starting...
W20260810 00:56:53.444639 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.010s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:56:53 dist-test-slave-jlzn krb5kdc[11201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:56:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 00:56:53.690932 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44509
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:54.005929 11220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:54.006172 11220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:54.006296 11220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:54.013132 11220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:54.013230 11220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:54.013304 11220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:54.013386 11220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:54.013445 11220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:54.022655 11220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44509
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:54.023684 11220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:54.024842 11220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:54.035530 11231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:54.035622 11227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:54.035888 11226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:54.036506 11220 server_base.cc:1034] running on GCE node
I20260810 00:56:54.037168 11220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:54.038367 11220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:54.039558 11220 hybrid_clock.cc:648] HybridClock initialized: now 1786323414039506 us; error 46 us; skew 500 ppm
I20260810 00:56:54.039901 11220 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:54 dist-test-slave-jlzn krb5kdc[11201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323414, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:54.042810 11220 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 00:56:54.044097 11220 webserver.cc:460] Webserver started at http://127.0.0.1:35199/ using document root <none> and password file <none>
I20260810 00:56:54.044706 11220 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:54.044821 11220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:54.045081 11220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:54.047778 11220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "c38c54dfd3294a6a8cf5e5aecd73ebde"
format_stamp: "Formatted at 2026-08-10 00:56:54 on dist-test-slave-jlzn"
server_key: "1ba13d3fee64c7c4e446cd10ac34a2fd"
server_key_iv: "3ad56d0bc57824d47f187473e8aec1da"
server_key_version: "encryptionkey@0"
I20260810 00:56:54.048527 11220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "c38c54dfd3294a6a8cf5e5aecd73ebde"
format_stamp: "Formatted at 2026-08-10 00:56:54 on dist-test-slave-jlzn"
server_key: "1ba13d3fee64c7c4e446cd10ac34a2fd"
server_key_iv: "3ad56d0bc57824d47f187473e8aec1da"
server_key_version: "encryptionkey@0"
I20260810 00:56:54.053272 11220 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:56:54.056411 11242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:54.057572 11220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.003s
I20260810 00:56:54.057730 11220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "c38c54dfd3294a6a8cf5e5aecd73ebde"
format_stamp: "Formatted at 2026-08-10 00:56:54 on dist-test-slave-jlzn"
server_key: "1ba13d3fee64c7c4e446cd10ac34a2fd"
server_key_iv: "3ad56d0bc57824d47f187473e8aec1da"
server_key_version: "encryptionkey@0"
I20260810 00:56:54.057899 11220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:54.063046 11220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:54.066902 11220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:54.067166 11220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:54.083071 11220 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44509
I20260810 00:56:54.083132 11326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44509 every 8 connection(s)
I20260810 00:56:54.084301 11220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:54.087590 11329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:54.091465 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 11220
I20260810 00:56:54.091629 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:54.092111 10629 external_mini_cluster.cc:1428] Setting key 318b1715c44eedeece6ce73a861e88d7
I20260810 00:56:54.097725 11329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde: Bootstrap starting.
I20260810 00:56:54.099256 11332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:54.093662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:46173 (local address 127.0.0.1:44509)
0810 00:56:54.094396 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:54.094406 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:54.094446 (+ 40us) server_negotiation.cc:407] Connection header received
0810 00:56:54.096376 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:54.096395 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:54.096667 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:54.096940 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:54.097875 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:54.097882 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:54.097893 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:54.097907 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:54.097964 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:54.097982 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:54.098147 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:54.098252 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 00:56:54.098414 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":311,"threads_started":1}
I20260810 00:56:54.100744 11329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:54.101832 11329 log.cc:824] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:54.102468 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:56:54.104620 11329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde: No bootstrap required, opened a new log
I20260810 00:56:54.109567 11332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:54.107284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:60305 (local address 127.0.0.1:44509)
0810 00:56:54.107460 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:54.107466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:54.107878 (+ 412us) server_negotiation.cc:407] Connection header received
0810 00:56:54.108030 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:54.108034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:54.108082 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:54.108172 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:54.109002 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:54.109006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:54.109009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:54.109022 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:54.109056 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:54.109070 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:54.109153 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:54.109261 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 00:56:54.109309 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 00:56:54.112926 11329 raft_consensus.cc:348] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38c54dfd3294a6a8cf5e5aecd73ebde" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44509 } }
I20260810 00:56:54.113236 11329 raft_consensus.cc:374] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:54.113328 11329 raft_consensus.cc:729] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c38c54dfd3294a6a8cf5e5aecd73ebde, State: Initialized, Role: FOLLOWER
I20260810 00:56:54.113807 11329 consensus_queue.cc:260] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38c54dfd3294a6a8cf5e5aecd73ebde" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44509 } }
I20260810 00:56:54.114023 11329 raft_consensus.cc:388] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:54.114148 11329 raft_consensus.cc:482] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:54.114302 11329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:54.116382 11329 raft_consensus.cc:504] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38c54dfd3294a6a8cf5e5aecd73ebde" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44509 } }
I20260810 00:56:54.116845 11329 leader_election.cc:302] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c38c54dfd3294a6a8cf5e5aecd73ebde; no voters:
I20260810 00:56:54.117331 11329 leader_election.cc:288] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:54.117435 11343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:54.118019 11343 raft_consensus.cc:686] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde [term 1 LEADER]: Becoming Leader. State: Replica: c38c54dfd3294a6a8cf5e5aecd73ebde, State: Running, Role: LEADER
I20260810 00:56:54.118505 11343 consensus_queue.cc:237] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38c54dfd3294a6a8cf5e5aecd73ebde" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44509 } }
I20260810 00:56:54.119081 11329 sys_catalog.cc:564] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:54.121202 11347 sys_catalog.cc:455] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde [sys.catalog]: SysCatalogTable state changed. Reason: New leader c38c54dfd3294a6a8cf5e5aecd73ebde. Latest consensus state: current_term: 1 leader_uuid: "c38c54dfd3294a6a8cf5e5aecd73ebde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38c54dfd3294a6a8cf5e5aecd73ebde" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44509 } } }
I20260810 00:56:54.121385 11346 sys_catalog.cc:455] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c38c54dfd3294a6a8cf5e5aecd73ebde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38c54dfd3294a6a8cf5e5aecd73ebde" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44509 } } }
I20260810 00:56:54.121644 11347 sys_catalog.cc:458] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:54.121675 11346 sys_catalog.cc:458] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:54.123471 11355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:54.127401 11355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:54.134521 11355 catalog_manager.cc:1347] Generated new cluster ID: 159c7bd371314f868d422119ce639132
I20260810 00:56:54.134616 11355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:54.154623 11355 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:54.155079 11355 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:54.160772 11355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c38c54dfd3294a6a8cf5e5aecd73ebde: Generated new TSK 0
I20260810 00:56:54.161399 11355 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:56:54.167093 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 11220
W20260810 00:56:54.168215 11322 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.68:46173
W20260810 00:56:54.168814 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 11220: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:56:54.168933 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 11220
2026-08-10T00:56:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1917 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:56:54 dist-test-slave-jlzn krb5kdc[11375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:56:54 dist-test-slave-jlzn krb5kdc[11375](info): set up 2 sockets
Aug 10 00:56:54 dist-test-slave-jlzn krb5kdc[11375](info): commencing operation
krb5kdc: starting...
W20260810 00:56:55.380740 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.005s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:56:55 dist-test-slave-jlzn krb5kdc[11375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:56:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:56:55.630199 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:44343
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:44343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:55.947610 11397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:55.947894 11397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:55.948037 11397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:55.954852 11397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:55.954959 11397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:55.955034 11397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:55.955118 11397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:55.955176 11397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:55.964323 11397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:44343
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:44343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:55.965400 11397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:55.966545 11397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:55.977139 11411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:55.977198 11409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:55.977231 11408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:55.978157 11397 server_base.cc:1034] running on GCE node
I20260810 00:56:55.978820 11397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:55.980043 11397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:55.981245 11397 hybrid_clock.cc:648] HybridClock initialized: now 1786323415981193 us; error 41 us; skew 500 ppm
I20260810 00:56:55.981545 11397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:55 dist-test-slave-jlzn krb5kdc[11375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323415, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:55.984462 11397 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:56:55.985693 11397 webserver.cc:460] Webserver started at http://127.10.97.126:44367/ using document root <none> and password file <none>
I20260810 00:56:55.986271 11397 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:55.986367 11397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:55.986596 11397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:55.989301 11397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "5b2d407ea7fa442f94af47436f853450"
format_stamp: "Formatted at 2026-08-10 00:56:55 on dist-test-slave-jlzn"
server_key: "f8bd14aabf44aa677a218fbc38205017"
server_key_iv: "d0136eaac812fc2f8bcb114521386359"
server_key_version: "encryptionkey@0"
I20260810 00:56:55.989953 11397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "5b2d407ea7fa442f94af47436f853450"
format_stamp: "Formatted at 2026-08-10 00:56:55 on dist-test-slave-jlzn"
server_key: "f8bd14aabf44aa677a218fbc38205017"
server_key_iv: "d0136eaac812fc2f8bcb114521386359"
server_key_version: "encryptionkey@0"
I20260810 00:56:55.994560 11397 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 00:56:55.997649 11419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:55.998765 11397 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:56:55.998903 11397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "5b2d407ea7fa442f94af47436f853450"
format_stamp: "Formatted at 2026-08-10 00:56:55 on dist-test-slave-jlzn"
server_key: "f8bd14aabf44aa677a218fbc38205017"
server_key_iv: "d0136eaac812fc2f8bcb114521386359"
server_key_version: "encryptionkey@0"
I20260810 00:56:55.999058 11397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:56.029486 11397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:56.033020 11397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:56.033255 11397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:56.049331 11397 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:44343
I20260810 00:56:56.049364 11496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:44343 every 8 connection(s)
I20260810 00:56:56.050518 11397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:56.053926 11498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:56.060894 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 11397
I20260810 00:56:56.061080 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:56.061606 10629 external_mini_cluster.cc:1428] Setting key d2973e80956e804d500ba596120a7a3d
I20260810 00:56:56.064239 11498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450: Bootstrap starting.
I20260810 00:56:56.067345 11498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:56.068511 11498 log.cc:824] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:56.071445 11498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450: No bootstrap required, opened a new log
Aug 10 00:56:56 dist-test-slave-jlzn krb5kdc[11375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:56:56.080816 11498 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b2d407ea7fa442f94af47436f853450" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44343 } }
I20260810 00:56:56.081141 11498 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:56.081228 11498 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2d407ea7fa442f94af47436f853450, State: Initialized, Role: FOLLOWER
I20260810 00:56:56.081749 11498 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b2d407ea7fa442f94af47436f853450" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44343 } }
I20260810 00:56:56.081969 11498 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:56.082108 11498 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:56.082288 11498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:56.085032 11502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:56.063226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34835 (local address 127.10.97.126:44343)
0810 00:56:56.063945 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:56.063955 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:56.063995 (+ 40us) server_negotiation.cc:407] Connection header received
0810 00:56:56.065668 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:56.065689 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:56.066036 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:56.066316 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:56.068112 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:56.069708 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:56.071391 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:56.072112 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:56.072244 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:56.076807 (+ 4563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:56.076817 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:56.076839 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:56.076880 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:56.080638 (+ 3758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:56.081406 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:56.081413 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:56.081419 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:56.081651 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:56.082108 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:56.082113 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:56.082116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:56.082811 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:56.083158 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:56.083499 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 00:56:56.083845 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":321,"threads_started":1}
I20260810 00:56:56.085397 11498 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b2d407ea7fa442f94af47436f853450" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44343 } }
I20260810 00:56:56.085933 11498 leader_election.cc:302] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b2d407ea7fa442f94af47436f853450; no voters:
I20260810 00:56:56.086550 11498 leader_election.cc:288] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:56.086644 11505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:56.087658 11505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450 [term 1 LEADER]: Becoming Leader. State: Replica: 5b2d407ea7fa442f94af47436f853450, State: Running, Role: LEADER
I20260810 00:56:56.088246 11505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b2d407ea7fa442f94af47436f853450" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44343 } }
I20260810 00:56:56.088624 11498 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:56.088788 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:56:56.090888 11507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b2d407ea7fa442f94af47436f853450" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b2d407ea7fa442f94af47436f853450" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44343 } } }
I20260810 00:56:56.091462 11508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b2d407ea7fa442f94af47436f853450. Latest consensus state: current_term: 1 leader_uuid: "5b2d407ea7fa442f94af47436f853450" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b2d407ea7fa442f94af47436f853450" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 44343 } } }
I20260810 00:56:56.092235 11507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:56.092324 11508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450 [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:56.095413 11520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:56.100595 11520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:56.111447 11520 catalog_manager.cc:1347] Generated new cluster ID: 775f124efa7f438098cd531667acab28
I20260810 00:56:56.111562 11520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:56.122466 11520 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:56.124504 11520 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:56.131268 11502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:56.112404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53629 (local address 127.10.97.126:44343)
0810 00:56:56.112560 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:56.112565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:56.112673 (+ 108us) server_negotiation.cc:407] Connection header received
0810 00:56:56.112942 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:56.112948 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:56.113029 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:56.113137 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:56.115219 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:56.116463 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:56.118425 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:56.119159 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:56.119254 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:56.123071 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:56.123079 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:56.123084 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:56:56.123126 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:56.126973 (+ 3847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:56.128023 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:56.128028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:56.128031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:56.128205 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:56:56.128791 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:56:56.128796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:56:56.128799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:56:56.129183 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:56.130490 (+ 1307us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:56.130688 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 00:56:56.130880 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 00:56:56.133765 11520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b2d407ea7fa442f94af47436f853450: Generated new TSK 0
I20260810 00:56:56.134975 11520 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:56:56.144500 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 11397
I20260810 00:56:56.145750 11491 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:56:56.231197 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 11397
W20260810 00:56:56.252004 11372 connection.cc:535] client connection to 127.10.97.126:44343 recv error: Network error: failed to read from TLS socket (remote: 127.10.97.126:44343): Connection reset by peer (error 104)
2026-08-10T00:56:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2061 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:56:56 dist-test-slave-jlzn krb5kdc[11547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:56:56 dist-test-slave-jlzn krb5kdc[11547](info): set up 2 sockets
Aug 10 00:56:56 dist-test-slave-jlzn krb5kdc[11547](info): commencing operation
krb5kdc: starting...
W20260810 00:56:57.424139 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:56:57 dist-test-slave-jlzn krb5kdc[11547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:56:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:56:57.676100 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:42751
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:42751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:57.990398 11565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:57.990664 11565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:57.990792 11565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:57.997632 11565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:57.997732 11565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:57.997810 11565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:57.997900 11565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:57.997956 11565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:58.007046 11565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:42751
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:42751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:58.008106 11565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:58.009253 11565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:56:58.020010 11578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:58.020072 11574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:56:58.020568 11573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:56:58.020834 11565 server_base.cc:1034] running on GCE node
I20260810 00:56:58.021523 11565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:56:58.022744 11565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:56:58.023882 11565 hybrid_clock.cc:648] HybridClock initialized: now 1786323418023834 us; error 40 us; skew 500 ppm
I20260810 00:56:58.024206 11565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:56:58 dist-test-slave-jlzn krb5kdc[11547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323418, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:56:58.027025 11565 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:56:58.028339 11565 webserver.cc:460] Webserver started at http://127.10.97.126:45147/ using document root <none> and password file <none>
I20260810 00:56:58.028934 11565 fs_manager.cc:359] Metadata directory not provided
I20260810 00:56:58.029045 11565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:56:58.029322 11565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:56:58.032132 11565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "a897940486f04e4c92a356e7625f609c"
format_stamp: "Formatted at 2026-08-10 00:56:58 on dist-test-slave-jlzn"
server_key: "9a94496adb3be54d7713144166857ea6"
server_key_iv: "1b4e410b4a547ad432c76a98a6c8d537"
server_key_version: "encryptionkey@0"
I20260810 00:56:58.032830 11565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "a897940486f04e4c92a356e7625f609c"
format_stamp: "Formatted at 2026-08-10 00:56:58 on dist-test-slave-jlzn"
server_key: "9a94496adb3be54d7713144166857ea6"
server_key_iv: "1b4e410b4a547ad432c76a98a6c8d537"
server_key_version: "encryptionkey@0"
I20260810 00:56:58.037452 11565 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:56:58.040573 11589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:56:58.041706 11565 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:56:58.041896 11565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "a897940486f04e4c92a356e7625f609c"
format_stamp: "Formatted at 2026-08-10 00:56:58 on dist-test-slave-jlzn"
server_key: "9a94496adb3be54d7713144166857ea6"
server_key_iv: "1b4e410b4a547ad432c76a98a6c8d537"
server_key_version: "encryptionkey@0"
I20260810 00:56:58.042062 11565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:56:58.060725 11565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:56:58.064240 11565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:56:58.064467 11565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:56:58.080088 11565 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:42751
I20260810 00:56:58.080099 11667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:42751 every 8 connection(s)
I20260810 00:56:58.081218 11565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:56:58.084520 11668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:56:58.085582 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 11565
I20260810 00:56:58.085739 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:56:58.086148 10629 external_mini_cluster.cc:1428] Setting key b0be6340f111cf675d393e6b4caf548c
I20260810 00:56:58.097801 11668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c: Bootstrap starting.
I20260810 00:56:58.099575 11674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:58.087730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39343 (local address 127.10.97.126:42751)
0810 00:56:58.088554 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:58.088572 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:58.088621 (+ 49us) server_negotiation.cc:407] Connection header received
0810 00:56:58.090423 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:58.090451 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:58.090913 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:58.091314 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:58.091795 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:58.093644 (+ 1849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:58.094901 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:58.095936 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:58.096085 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:58.097097 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:58.097107 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:58.097122 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:58.097761 (+ 639us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:58.097856 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:58.097889 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:58.098036 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:58.098253 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 00:56:58.098397 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":349,"threads_started":1}
I20260810 00:56:58.101285 11668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c: Neither blocks nor log segments found. Creating new log.
I20260810 00:56:58.102452 11668 log.cc:824] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c: Log is configured to *not* fsync() on all Append() calls
I20260810 00:56:58.103196 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:56:58.104933 11668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c: No bootstrap required, opened a new log
I20260810 00:56:58.112478 11668 raft_consensus.cc:348] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a897940486f04e4c92a356e7625f609c" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42751 } }
I20260810 00:56:58.112872 11668 raft_consensus.cc:374] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:56:58.112960 11674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:56:58.107205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34573 (local address 127.10.97.126:42751)
0810 00:56:58.107335 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 00:56:58.107339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:56:58.107860 (+ 521us) server_negotiation.cc:407] Connection header received
0810 00:56:58.108012 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:56:58.108017 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:56:58.108075 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:56:58.108169 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:56:58.108611 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:58.109351 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:58.110657 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:56:58.111115 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:56:58.111232 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:56:58.112296 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:56:58.112301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:56:58.112304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:56:58.112319 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:56:58.112366 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:56:58.112380 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:56:58.112474 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 00:56:58.112647 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 00:56:58.112689 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 00:56:58.112998 11668 raft_consensus.cc:729] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a897940486f04e4c92a356e7625f609c, State: Initialized, Role: FOLLOWER
I20260810 00:56:58.113607 11668 consensus_queue.cc:260] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a897940486f04e4c92a356e7625f609c" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42751 } }
I20260810 00:56:58.113847 11668 raft_consensus.cc:388] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:56:58.113976 11668 raft_consensus.cc:482] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:56:58.114162 11668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:56:58.116178 11668 raft_consensus.cc:504] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a897940486f04e4c92a356e7625f609c" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42751 } }
I20260810 00:56:58.116613 11668 leader_election.cc:302] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a897940486f04e4c92a356e7625f609c; no voters:
I20260810 00:56:58.117137 11668 leader_election.cc:288] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:56:58.117238 11684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:56:58.117852 11684 raft_consensus.cc:686] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c [term 1 LEADER]: Becoming Leader. State: Replica: a897940486f04e4c92a356e7625f609c, State: Running, Role: LEADER
I20260810 00:56:58.118352 11684 consensus_queue.cc:237] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a897940486f04e4c92a356e7625f609c" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42751 } }
I20260810 00:56:58.118935 11668 sys_catalog.cc:564] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:56:58.120996 11688 sys_catalog.cc:455] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a897940486f04e4c92a356e7625f609c. Latest consensus state: current_term: 1 leader_uuid: "a897940486f04e4c92a356e7625f609c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a897940486f04e4c92a356e7625f609c" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42751 } } }
I20260810 00:56:58.121057 11686 sys_catalog.cc:455] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a897940486f04e4c92a356e7625f609c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a897940486f04e4c92a356e7625f609c" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 42751 } } }
I20260810 00:56:58.121436 11686 sys_catalog.cc:458] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:58.121441 11688 sys_catalog.cc:458] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c [sys.catalog]: This master's current role is: LEADER
I20260810 00:56:58.123430 11695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:56:58.128664 11695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:56:58.136593 11695 catalog_manager.cc:1347] Generated new cluster ID: 2d4902c968384188ae16b87778a2dbeb
I20260810 00:56:58.136700 11695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:56:58.156360 11695 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:56:58.157529 11695 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:56:58.173900 11695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a897940486f04e4c92a356e7625f609c: Generated new TSK 0
I20260810 00:56:58.174635 11695 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:56:58.191427 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 11565
W20260810 00:56:58.192929 11661 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39343
W20260810 00:56:58.193614 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 11565: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:56:58.193730 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 11565
W20260810 00:56:58.216821 11544 connection.cc:535] client connection to 127.10.97.126:42751 recv error: Network error: failed to read from TLS socket (remote: 127.10.97.126:42751): Connection reset by peer (error 104)
2026-08-10T00:56:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1964 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:56:58 dist-test-slave-jlzn krb5kdc[11723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:56:58 dist-test-slave-jlzn krb5kdc[11723](info): set up 2 sockets
Aug 10 00:56:58 dist-test-slave-jlzn krb5kdc[11723](info): commencing operation
krb5kdc: starting...
W20260810 00:56:59.396411 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.008s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:56:59 dist-test-slave-jlzn krb5kdc[11723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:56:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:56:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:56:59.650197 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:37131
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:39953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:37131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:56:59.970285 11741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:56:59.970551 11741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:56:59.970687 11741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:56:59.977496 11741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:56:59.977603 11741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:56:59.977694 11741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:56:59.977775 11741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:56:59.977835 11741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:56:59.986937 11741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:39953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:37131
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:37131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:56:59.988097 11741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:56:59.989285 11741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:00.000188 11752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:00.000252 11756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:00.000227 11751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:00.001225 11741 server_base.cc:1034] running on GCE node
I20260810 00:57:00.001878 11741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:00.003062 11741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:00.004240 11741 hybrid_clock.cc:648] HybridClock initialized: now 1786323420004194 us; error 39 us; skew 500 ppm
I20260810 00:57:00.004537 11741 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:00 dist-test-slave-jlzn krb5kdc[11723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323420, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:00.007366 11741 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:57:00.008661 11741 webserver.cc:460] Webserver started at http://127.10.97.126:42279/ using document root <none> and password file <none>
I20260810 00:57:00.009234 11741 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:00.009334 11741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:00.009567 11741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:00.012331 11741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "4cfe95f0ffa740f6939fd63ccd1c8283"
format_stamp: "Formatted at 2026-08-10 00:57:00 on dist-test-slave-jlzn"
server_key: "dbcb429d0da339b5f781c2ef7a45627d"
server_key_iv: "b9375071cb5b79c48ad3f2dc901e0880"
server_key_version: "encryptionkey@0"
I20260810 00:57:00.013036 11741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "4cfe95f0ffa740f6939fd63ccd1c8283"
format_stamp: "Formatted at 2026-08-10 00:57:00 on dist-test-slave-jlzn"
server_key: "dbcb429d0da339b5f781c2ef7a45627d"
server_key_iv: "b9375071cb5b79c48ad3f2dc901e0880"
server_key_version: "encryptionkey@0"
I20260810 00:57:00.017719 11741 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:57:00.020823 11772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:00.021937 11741 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.003s
I20260810 00:57:00.022086 11741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "4cfe95f0ffa740f6939fd63ccd1c8283"
format_stamp: "Formatted at 2026-08-10 00:57:00 on dist-test-slave-jlzn"
server_key: "dbcb429d0da339b5f781c2ef7a45627d"
server_key_iv: "b9375071cb5b79c48ad3f2dc901e0880"
server_key_version: "encryptionkey@0"
I20260810 00:57:00.022264 11741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:00.027328 11741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:00.030752 11741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:00.030997 11741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:00.047043 11741 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:37131
I20260810 00:57:00.047079 11852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:37131 every 8 connection(s)
I20260810 00:57:00.048274 11741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:57:00.050357 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 11741
I20260810 00:57:00.050520 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:57:00.050940 10629 external_mini_cluster.cc:1428] Setting key f1e168b72789139fddabe8c5506f4857
I20260810 00:57:00.051760 11853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:00.058611 11857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:00.052526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51955 (local address 127.10.97.126:37131)
0810 00:57:00.053302 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:00.053316 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:00.053368 (+ 52us) server_negotiation.cc:407] Connection header received
0810 00:57:00.055059 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:00.055085 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:00.055486 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:00.055913 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:00.056822 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:00.056830 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:00.056845 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:57:00.056863 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:00.056956 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:57:00.056987 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:00.057182 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:00.057316 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 00:57:00.057494 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":318,"threads_started":1}
I20260810 00:57:00.061920 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:57:00.064648 11853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283: Bootstrap starting.
I20260810 00:57:00.067834 11853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:00.068411 11857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:00.066108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43699 (local address 127.10.97.126:37131)
0810 00:57:00.066269 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:00.066275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:00.066660 (+ 385us) server_negotiation.cc:407] Connection header received
0810 00:57:00.066793 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:00.066798 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:00.066843 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:00.066918 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:00.067733 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:00.067739 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:00.067743 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:57:00.067761 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:00.067841 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:57:00.067861 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:00.067968 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:00.068103 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 00:57:00.068150 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 00:57:00.069072 11853 log.cc:824] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:00.071472 11853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283: No bootstrap required, opened a new log
I20260810 00:57:00.078449 11853 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cfe95f0ffa740f6939fd63ccd1c8283" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 37131 } }
I20260810 00:57:00.078753 11853 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:00.078840 11853 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cfe95f0ffa740f6939fd63ccd1c8283, State: Initialized, Role: FOLLOWER
I20260810 00:57:00.079371 11853 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cfe95f0ffa740f6939fd63ccd1c8283" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 37131 } }
I20260810 00:57:00.079574 11853 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:57:00.079702 11853 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:57:00.079876 11853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:00.082005 11853 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cfe95f0ffa740f6939fd63ccd1c8283" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 37131 } }
I20260810 00:57:00.082437 11853 leader_election.cc:302] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cfe95f0ffa740f6939fd63ccd1c8283; no voters:
I20260810 00:57:00.082931 11853 leader_election.cc:288] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:57:00.083026 11869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:00.083632 11869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283 [term 1 LEADER]: Becoming Leader. State: Replica: 4cfe95f0ffa740f6939fd63ccd1c8283, State: Running, Role: LEADER
I20260810 00:57:00.084146 11869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cfe95f0ffa740f6939fd63ccd1c8283" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 37131 } }
I20260810 00:57:00.084729 11853 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:57:00.086647 11872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cfe95f0ffa740f6939fd63ccd1c8283. Latest consensus state: current_term: 1 leader_uuid: "4cfe95f0ffa740f6939fd63ccd1c8283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cfe95f0ffa740f6939fd63ccd1c8283" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 37131 } } }
I20260810 00:57:00.086783 11870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cfe95f0ffa740f6939fd63ccd1c8283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cfe95f0ffa740f6939fd63ccd1c8283" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 37131 } } }
I20260810 00:57:00.087028 11872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283 [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:00.087028 11870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283 [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:00.088761 11881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:57:00.092810 11881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:57:00.101451 11881 catalog_manager.cc:1347] Generated new cluster ID: 46185821240e4a949268cb18274b4f99
I20260810 00:57:00.101538 11881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:57:00.112998 11881 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:57:00.113332 11881 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:57:00.137589 11881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cfe95f0ffa740f6939fd63ccd1c8283: Generated new TSK 0
I20260810 00:57:00.138322 11881 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:57:00.151217 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 11741
W20260810 00:57:00.152235 11846 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51955
W20260810 00:57:00.152850 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 11741: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:57:00.152969 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 11741
2026-08-10T00:57:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1961 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:57:00 dist-test-slave-jlzn krb5kdc[11901](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:57:00 dist-test-slave-jlzn krb5kdc[11901](info): set up 2 sockets
Aug 10 00:57:00 dist-test-slave-jlzn krb5kdc[11901](info): commencing operation
krb5kdc: starting...
W20260810 00:57:01.348966 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.001s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:57:01 dist-test-slave-jlzn krb5kdc[11901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:57:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:57:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:57:01.599012 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:36895
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:41395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:36895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:01.911360 11928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:01.911628 11928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:01.911762 11928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:01.918582 11928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:57:01.918690 11928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:01.918769 11928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:01.918857 11928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:57:01.918915 11928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:57:01.929090 11928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:41395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:36895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:36895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:01.930183 11928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:01.931392 11928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:01.942257 11939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:01.942329 11940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:01.942296 11944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:01.943018 11928 server_base.cc:1034] running on GCE node
I20260810 00:57:01.943714 11928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:01.944976 11928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:01.946179 11928 hybrid_clock.cc:648] HybridClock initialized: now 1786323421946135 us; error 31 us; skew 500 ppm
I20260810 00:57:01.946563 11928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:01 dist-test-slave-jlzn krb5kdc[11901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323421, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:01.953263 11928 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:57:01.954560 11928 webserver.cc:460] Webserver started at http://127.10.97.126:45283/ using document root <none> and password file <none>
I20260810 00:57:01.955155 11928 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:01.955267 11928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:01.955534 11928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:01.958331 11928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "f38bda96ee3f4c2bbcd35b09cfb88718"
format_stamp: "Formatted at 2026-08-10 00:57:01 on dist-test-slave-jlzn"
server_key: "bea6519f9210d5d4391af8fc1d518f00"
server_key_iv: "c166d02db92458f1c20e49c4e9b22df2"
server_key_version: "encryptionkey@0"
I20260810 00:57:01.959031 11928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "f38bda96ee3f4c2bbcd35b09cfb88718"
format_stamp: "Formatted at 2026-08-10 00:57:01 on dist-test-slave-jlzn"
server_key: "bea6519f9210d5d4391af8fc1d518f00"
server_key_iv: "c166d02db92458f1c20e49c4e9b22df2"
server_key_version: "encryptionkey@0"
I20260810 00:57:01.963677 11928 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260810 00:57:01.966899 11960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:01.968165 11928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 00:57:01.968310 11928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "f38bda96ee3f4c2bbcd35b09cfb88718"
format_stamp: "Formatted at 2026-08-10 00:57:01 on dist-test-slave-jlzn"
server_key: "bea6519f9210d5d4391af8fc1d518f00"
server_key_iv: "c166d02db92458f1c20e49c4e9b22df2"
server_key_version: "encryptionkey@0"
I20260810 00:57:01.968469 11928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:01.989502 11928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:01.993005 11928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:01.993244 11928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:02.009156 11928 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:36895
I20260810 00:57:02.009191 12034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:36895 every 8 connection(s)
I20260810 00:57:02.010325 11928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:57:02.013654 12035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:02.018656 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 11928
I20260810 00:57:02.018827 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:57:02.019361 10629 external_mini_cluster.cc:1428] Setting key 948c7bb5b83afffe1330d2d6377ba52a
I20260810 00:57:02.023857 12035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718: Bootstrap starting.
I20260810 00:57:02.027132 12035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:02.028309 12035 log.cc:824] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:02.031275 12035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718: No bootstrap required, opened a new log
Aug 10 00:57:02 dist-test-slave-jlzn krb5kdc[11901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:57:02.039541 12035 raft_consensus.cc:348] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38bda96ee3f4c2bbcd35b09cfb88718" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 36895 } }
I20260810 00:57:02.039999 12035 raft_consensus.cc:374] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:02.040117 12035 raft_consensus.cc:729] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f38bda96ee3f4c2bbcd35b09cfb88718, State: Initialized, Role: FOLLOWER
I20260810 00:57:02.040670 12035 consensus_queue.cc:260] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38bda96ee3f4c2bbcd35b09cfb88718" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 36895 } }
I20260810 00:57:02.040901 12035 raft_consensus.cc:388] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:57:02.041029 12035 raft_consensus.cc:482] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:57:02.041193 12035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:02.041555 12041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:02.021081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38391 (local address 127.10.97.126:36895)
0810 00:57:02.021867 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:02.021878 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:02.021918 (+ 40us) server_negotiation.cc:407] Connection header received
0810 00:57:02.023496 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:02.023516 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:02.023884 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:02.024208 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:02.025937 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:02.027133 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:02.028877 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:02.029787 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:02.029911 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:02.034497 (+ 4586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:02.034504 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:02.034520 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:02.034558 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:02.037525 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:02.038355 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:02.038361 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:02.038366 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:02.038522 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:02.038891 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:02.038894 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:02.038896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:02.039351 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:02.039548 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:02.040053 (+ 505us) server_negotiation.cc:299] Negotiation successful
0810 00:57:02.040439 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":364,"threads_started":1}
I20260810 00:57:02.043426 12035 raft_consensus.cc:504] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38bda96ee3f4c2bbcd35b09cfb88718" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 36895 } }
I20260810 00:57:02.043936 12035 leader_election.cc:302] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f38bda96ee3f4c2bbcd35b09cfb88718; no voters:
I20260810 00:57:02.044520 12035 leader_election.cc:288] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:57:02.044613 12044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:02.044833 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:57:02.046571 12035 sys_catalog.cc:564] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:57:02.051388 12044 raft_consensus.cc:686] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718 [term 1 LEADER]: Becoming Leader. State: Replica: f38bda96ee3f4c2bbcd35b09cfb88718, State: Running, Role: LEADER
I20260810 00:57:02.051981 12044 consensus_queue.cc:237] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38bda96ee3f4c2bbcd35b09cfb88718" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 36895 } }
I20260810 00:57:02.055094 12047 sys_catalog.cc:455] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f38bda96ee3f4c2bbcd35b09cfb88718" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38bda96ee3f4c2bbcd35b09cfb88718" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 36895 } } }
I20260810 00:57:02.055502 12047 sys_catalog.cc:458] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718 [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:02.055986 12048 sys_catalog.cc:455] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f38bda96ee3f4c2bbcd35b09cfb88718. Latest consensus state: current_term: 1 leader_uuid: "f38bda96ee3f4c2bbcd35b09cfb88718" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38bda96ee3f4c2bbcd35b09cfb88718" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 36895 } } }
I20260810 00:57:02.056264 12048 sys_catalog.cc:458] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718 [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:02.060722 12069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:57:02.065584 12069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:57:02.073422 12041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:02.054128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57719 (local address 127.10.97.126:36895)
0810 00:57:02.054290 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:02.054296 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:02.057824 (+ 3528us) server_negotiation.cc:407] Connection header received
0810 00:57:02.058534 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:02.058537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:02.058593 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:02.058670 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:02.060082 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:02.061158 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:02.062888 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:02.063620 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:02.063766 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:02.066794 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:02.066800 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:02.066805 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:02.066842 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:02.070542 (+ 3700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:02.071372 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:02.071377 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:02.071380 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:02.071551 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:02.072008 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:02.072012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:02.072014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:02.072350 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:02.072576 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:02.072964 (+ 388us) server_negotiation.cc:299] Negotiation successful
0810 00:57:02.073153 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 00:57:02.080600 12069 catalog_manager.cc:1347] Generated new cluster ID: 03d2fa2c77c24806981222773bad2f29
I20260810 00:57:02.080703 12069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:57:02.097115 12069 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:57:02.098232 12069 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:57:02.115115 12069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f38bda96ee3f4c2bbcd35b09cfb88718: Generated new TSK 0
I20260810 00:57:02.115866 12069 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:57:02.132450 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 11928
I20260810 00:57:02.134778 12028 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:02.223909 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 11928
2026-08-10T00:57:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2066 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:57:02 dist-test-slave-jlzn krb5kdc[12084](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:57:02 dist-test-slave-jlzn krb5kdc[12084](info): set up 2 sockets
Aug 10 00:57:02 dist-test-slave-jlzn krb5kdc[12084](info): commencing operation
krb5kdc: starting...
W20260810 00:57:03.413321 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:57:03 dist-test-slave-jlzn krb5kdc[12084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:57:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:57:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:57:03.666086 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:34635
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:41099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:34635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:03.986073 12107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:03.986347 12107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:03.986450 12107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:03.993389 12107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:57:03.993502 12107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:03.993556 12107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:03.993615 12107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:57:03.993659 12107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:57:04.002949 12107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:41099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:34635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:34635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:04.004048 12107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:04.005240 12107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:04.016049 12113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:04.016088 12114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:04.016216 12118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:04.017068 12107 server_base.cc:1034] running on GCE node
I20260810 00:57:04.017774 12107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:04.018950 12107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:04.020138 12107 hybrid_clock.cc:648] HybridClock initialized: now 1786323424020100 us; error 28 us; skew 500 ppm
I20260810 00:57:04.020460 12107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:04 dist-test-slave-jlzn krb5kdc[12084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323424, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:04.023296 12107 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:57:04.024668 12107 webserver.cc:460] Webserver started at http://127.10.97.126:33963/ using document root <none> and password file <none>
I20260810 00:57:04.025279 12107 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:04.025398 12107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:04.025666 12107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:04.028493 12107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "a5683457efb24c568fa23f76bd5bdf96"
format_stamp: "Formatted at 2026-08-10 00:57:04 on dist-test-slave-jlzn"
server_key: "8ef6f4ec741955985366802d508603a1"
server_key_iv: "2c3634dc37dc7328090216a203f5ac38"
server_key_version: "encryptionkey@0"
I20260810 00:57:04.029206 12107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "a5683457efb24c568fa23f76bd5bdf96"
format_stamp: "Formatted at 2026-08-10 00:57:04 on dist-test-slave-jlzn"
server_key: "8ef6f4ec741955985366802d508603a1"
server_key_iv: "2c3634dc37dc7328090216a203f5ac38"
server_key_version: "encryptionkey@0"
I20260810 00:57:04.033984 12107 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:57:04.037189 12130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:04.038372 12107 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:57:04.038534 12107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "a5683457efb24c568fa23f76bd5bdf96"
format_stamp: "Formatted at 2026-08-10 00:57:04 on dist-test-slave-jlzn"
server_key: "8ef6f4ec741955985366802d508603a1"
server_key_iv: "2c3634dc37dc7328090216a203f5ac38"
server_key_version: "encryptionkey@0"
I20260810 00:57:04.038702 12107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:04.064286 12107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:04.067823 12107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:04.068070 12107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:04.085120 12107 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:34635
I20260810 00:57:04.085144 12208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:34635 every 8 connection(s)
I20260810 00:57:04.086337 12107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:57:04.087278 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 12107
I20260810 00:57:04.087462 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:57:04.088052 10629 external_mini_cluster.cc:1428] Setting key a4dcdec65e337fb2794caa077aac298b
I20260810 00:57:04.090618 12210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:04.100173 12214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:04.089767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37049 (local address 127.10.97.126:34635)
0810 00:57:04.090495 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:04.090506 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:04.090544 (+ 38us) server_negotiation.cc:407] Connection header received
0810 00:57:04.092230 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:04.092252 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:04.092586 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:04.092944 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:04.093456 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:04.094753 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:04.096081 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:04.096718 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:04.096878 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:04.098030 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:04.098037 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:04.098049 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:57:04.098570 (+ 521us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:04.098644 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:57:04.098676 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:04.098800 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:04.099021 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 00:57:04.099260 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":327,"threads_started":1}
I20260810 00:57:04.100809 12210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96: Bootstrap starting.
I20260810 00:57:04.103451 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:57:04.103791 12210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:04.104732 12210 log.cc:824] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:04.107121 12210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96: No bootstrap required, opened a new log
I20260810 00:57:04.113780 12214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:04.107381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41219 (local address 127.10.97.126:34635)
0810 00:57:04.107547 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:04.107557 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:04.108096 (+ 539us) server_negotiation.cc:407] Connection header received
0810 00:57:04.108271 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:04.108275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:04.108328 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:04.108402 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:04.109001 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:04.109735 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:04.111515 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:04.112031 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:04.112129 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:04.113144 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:04.113148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:04.113151 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:57:04.113166 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:04.113202 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:57:04.113217 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:04.113290 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:04.113410 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 00:57:04.113472 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 00:57:04.116050 12210 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5683457efb24c568fa23f76bd5bdf96" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 34635 } }
I20260810 00:57:04.116382 12210 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:04.116475 12210 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5683457efb24c568fa23f76bd5bdf96, State: Initialized, Role: FOLLOWER
I20260810 00:57:04.116957 12210 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5683457efb24c568fa23f76bd5bdf96" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 34635 } }
I20260810 00:57:04.117162 12210 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:57:04.117290 12210 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:57:04.117441 12210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:04.119637 12210 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5683457efb24c568fa23f76bd5bdf96" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 34635 } }
I20260810 00:57:04.120105 12210 leader_election.cc:302] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5683457efb24c568fa23f76bd5bdf96; no voters:
I20260810 00:57:04.120600 12210 leader_election.cc:288] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:57:04.120705 12230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:04.121377 12230 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96 [term 1 LEADER]: Becoming Leader. State: Replica: a5683457efb24c568fa23f76bd5bdf96, State: Running, Role: LEADER
I20260810 00:57:04.121879 12230 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5683457efb24c568fa23f76bd5bdf96" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 34635 } }
I20260810 00:57:04.122447 12210 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:57:04.124300 12232 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5683457efb24c568fa23f76bd5bdf96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5683457efb24c568fa23f76bd5bdf96" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 34635 } } }
I20260810 00:57:04.124715 12232 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96 [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:04.124680 12233 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5683457efb24c568fa23f76bd5bdf96. Latest consensus state: current_term: 1 leader_uuid: "a5683457efb24c568fa23f76bd5bdf96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5683457efb24c568fa23f76bd5bdf96" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 34635 } } }
I20260810 00:57:04.124958 12233 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96 [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:04.126346 12242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:57:04.130976 12242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:57:04.139531 12242 catalog_manager.cc:1347] Generated new cluster ID: 399c40e90c234961be1bb612c4891445
I20260810 00:57:04.139621 12242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:57:04.163699 12242 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:57:04.165509 12242 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:57:04.193099 12242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5683457efb24c568fa23f76bd5bdf96: Generated new TSK 0
I20260810 00:57:04.194027 12242 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:57:04.200848 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 12107
W20260810 00:57:04.202080 12203 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37049
W20260810 00:57:04.202723 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 12107: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:57:04.202857 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 12107
2026-08-10T00:57:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1980 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:57:04 dist-test-slave-jlzn krb5kdc[12263](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:57:04 dist-test-slave-jlzn krb5kdc[12263](info): set up 2 sockets
Aug 10 00:57:04 dist-test-slave-jlzn krb5kdc[12263](info): commencing operation
krb5kdc: starting...
W20260810 00:57:05.398540 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.006s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:57:05 dist-test-slave-jlzn krb5kdc[12263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:57:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:57:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:57:05.656924 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:40705
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:43937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:40705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:05.973070 12279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:05.973315 12279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:05.973408 12279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:05.980265 12279 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:57:05.980357 12279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:05.980408 12279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:05.980466 12279 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:57:05.980510 12279 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:57:05.989609 12279 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:43937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:40705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:40705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:05.990646 12279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:05.991844 12279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:06.002367 12292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:06.002442 12295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:06.002408 12293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:06.003499 12279 server_base.cc:1034] running on GCE node
I20260810 00:57:06.004310 12279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:06.005527 12279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:06.006719 12279 hybrid_clock.cc:648] HybridClock initialized: now 1786323426006680 us; error 30 us; skew 500 ppm
I20260810 00:57:06.007032 12279 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:06 dist-test-slave-jlzn krb5kdc[12263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323426, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:06.009918 12279 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:57:06.011220 12279 webserver.cc:460] Webserver started at http://127.10.97.126:41977/ using document root <none> and password file <none>
I20260810 00:57:06.011883 12279 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:06.012005 12279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:06.012291 12279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:06.015046 12279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "040304d3880644dfbf6f0117f7074ef4"
format_stamp: "Formatted at 2026-08-10 00:57:06 on dist-test-slave-jlzn"
server_key: "0d71547df5ab8601d970378fd5a115ba"
server_key_iv: "01daafa324cba6e371d5044f1646b2bb"
server_key_version: "encryptionkey@0"
I20260810 00:57:06.015760 12279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "040304d3880644dfbf6f0117f7074ef4"
format_stamp: "Formatted at 2026-08-10 00:57:06 on dist-test-slave-jlzn"
server_key: "0d71547df5ab8601d970378fd5a115ba"
server_key_iv: "01daafa324cba6e371d5044f1646b2bb"
server_key_version: "encryptionkey@0"
I20260810 00:57:06.020560 12279 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 00:57:06.023643 12305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:06.024827 12279 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:57:06.024986 12279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "040304d3880644dfbf6f0117f7074ef4"
format_stamp: "Formatted at 2026-08-10 00:57:06 on dist-test-slave-jlzn"
server_key: "0d71547df5ab8601d970378fd5a115ba"
server_key_iv: "01daafa324cba6e371d5044f1646b2bb"
server_key_version: "encryptionkey@0"
I20260810 00:57:06.025159 12279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:06.030272 12279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:06.033771 12279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:06.034027 12279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:06.050000 12279 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:40705
I20260810 00:57:06.050034 12374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:40705 every 8 connection(s)
I20260810 00:57:06.051199 12279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:57:06.054536 12375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:06.058243 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 12279
I20260810 00:57:06.058401 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:57:06.058799 10629 external_mini_cluster.cc:1428] Setting key 275b7e57df81ac2bf35a1da5ff8b3f90
I20260810 00:57:06.066332 12379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:06.060351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57583 (local address 127.10.97.126:40705)
0810 00:57:06.061055 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:06.061069 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:06.061119 (+ 50us) server_negotiation.cc:407] Connection header received
0810 00:57:06.062791 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:06.062817 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:06.063222 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:06.063628 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:06.064580 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:06.064589 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:06.064606 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:57:06.064641 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:06.064709 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:57:06.064734 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:06.064958 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:06.065086 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 00:57:06.065258 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":291,"threads_started":1}
I20260810 00:57:06.067117 12375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4: Bootstrap starting.
I20260810 00:57:06.069563 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:57:06.070116 12375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:06.071262 12375 log.cc:824] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:06.074301 12375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4: No bootstrap required, opened a new log
I20260810 00:57:06.076514 12379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:06.074318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54963 (local address 127.10.97.126:40705)
0810 00:57:06.074517 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:06.074523 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:06.074932 (+ 409us) server_negotiation.cc:407] Connection header received
0810 00:57:06.075041 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:06.075046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:06.075095 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:06.075195 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:06.075956 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:06.075961 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:06.075965 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 00:57:06.075983 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:06.076028 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 00:57:06.076047 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:06.076157 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:06.076203 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 00:57:06.076254 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260810 00:57:06.081826 12375 raft_consensus.cc:348] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040304d3880644dfbf6f0117f7074ef4" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 40705 } }
I20260810 00:57:06.082147 12375 raft_consensus.cc:374] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:57:06.082222 12375 raft_consensus.cc:729] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 040304d3880644dfbf6f0117f7074ef4, State: Initialized, Role: FOLLOWER
I20260810 00:57:06.082649 12375 consensus_queue.cc:260] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040304d3880644dfbf6f0117f7074ef4" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 40705 } }
I20260810 00:57:06.082847 12375 raft_consensus.cc:388] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:57:06.082937 12375 raft_consensus.cc:482] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:57:06.083076 12375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:06.085278 12375 raft_consensus.cc:504] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040304d3880644dfbf6f0117f7074ef4" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 40705 } }
I20260810 00:57:06.085678 12375 leader_election.cc:302] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 040304d3880644dfbf6f0117f7074ef4; no voters:
I20260810 00:57:06.086158 12375 leader_election.cc:288] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:57:06.086253 12391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:06.086861 12391 raft_consensus.cc:686] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4 [term 1 LEADER]: Becoming Leader. State: Replica: 040304d3880644dfbf6f0117f7074ef4, State: Running, Role: LEADER
I20260810 00:57:06.087384 12391 consensus_queue.cc:237] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040304d3880644dfbf6f0117f7074ef4" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 40705 } }
I20260810 00:57:06.088171 12375 sys_catalog.cc:564] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:57:06.090112 12392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "040304d3880644dfbf6f0117f7074ef4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040304d3880644dfbf6f0117f7074ef4" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 40705 } } }
I20260810 00:57:06.090665 12392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4 [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:06.091317 12393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 040304d3880644dfbf6f0117f7074ef4. Latest consensus state: current_term: 1 leader_uuid: "040304d3880644dfbf6f0117f7074ef4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040304d3880644dfbf6f0117f7074ef4" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 40705 } } }
I20260810 00:57:06.091562 12393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4 [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:06.092232 12405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:57:06.096738 12405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:57:06.105464 12405 catalog_manager.cc:1347] Generated new cluster ID: a667cd42e6b540d3b41e3c39b880bbee
I20260810 00:57:06.105554 12405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:57:06.123291 12405 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:57:06.123646 12405 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:57:06.131450 12405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 040304d3880644dfbf6f0117f7074ef4: Generated new TSK 0
I20260810 00:57:06.132114 12405 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:57:06.143235 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 12279
W20260810 00:57:06.144407 12369 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57583
W20260810 00:57:06.144986 10629 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 12279: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 00:57:06.145107 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 12279
2026-08-10T00:57:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1943 ms)
[----------] 18 tests from AuthTokenIssuingTest (35907 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:57:06 dist-test-slave-jlzn krb5kdc[12424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:57:06 dist-test-slave-jlzn krb5kdc[12424](info): set up 2 sockets
Aug 10 00:57:06 dist-test-slave-jlzn krb5kdc[12424](info): commencing operation
krb5kdc: starting...
W20260810 00:57:07.351298 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:57:07 dist-test-slave-jlzn krb5kdc[12424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:57:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:57:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:57:07.606762 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:41279
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:38199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:41279,127.10.97.125:43875,127.10.97.124:46153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:07.927253 12444 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 00:57:07.927572 12444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:07.927671 12444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:07.927791 12444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:07.934885 12444 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:57:07.934962 12444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:07.935042 12444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:07.935113 12444 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:57:07.935195 12444 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:57:07.946125 12444 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:38199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:41279,127.10.97.125:43875,127.10.97.124:46153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:41279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:07.947256 12444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:07.948501 12444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:07.959375 12459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:07.959465 12454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:07.959435 12455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:07.960191 12444 server_base.cc:1034] running on GCE node
I20260810 00:57:07.960911 12444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:07.962100 12444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:07.963323 12444 hybrid_clock.cc:648] HybridClock initialized: now 1786323427963261 us; error 35 us; skew 500 ppm
I20260810 00:57:07.963636 12444 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:07 dist-test-slave-jlzn krb5kdc[12424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323427, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:07.969611 12444 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:57:07.970978 12444 webserver.cc:460] Webserver started at http://127.10.97.126:44715/ using document root <none> and password file <none>
I20260810 00:57:07.971598 12444 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:07.971698 12444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:07.972023 12444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:07.974750 12444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "ad5dbcb943354bc38c8e62e867ce513f"
format_stamp: "Formatted at 2026-08-10 00:57:07 on dist-test-slave-jlzn"
server_key: "d51310e4e66f7367beb0ea81f1c9e4f5"
server_key_iv: "7df36cdcef2a61a134b75dde1a81a4e3"
server_key_version: "encryptionkey@0"
I20260810 00:57:07.975462 12444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "ad5dbcb943354bc38c8e62e867ce513f"
format_stamp: "Formatted at 2026-08-10 00:57:07 on dist-test-slave-jlzn"
server_key: "d51310e4e66f7367beb0ea81f1c9e4f5"
server_key_iv: "7df36cdcef2a61a134b75dde1a81a4e3"
server_key_version: "encryptionkey@0"
I20260810 00:57:07.980168 12444 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260810 00:57:07.983284 12467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:07.984495 12444 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:57:07.984666 12444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "ad5dbcb943354bc38c8e62e867ce513f"
format_stamp: "Formatted at 2026-08-10 00:57:07 on dist-test-slave-jlzn"
server_key: "d51310e4e66f7367beb0ea81f1c9e4f5"
server_key_iv: "7df36cdcef2a61a134b75dde1a81a4e3"
server_key_version: "encryptionkey@0"
I20260810 00:57:07.984851 12444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:08.000789 12444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:08.004266 12444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:08.004493 12444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:08.021205 12444 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:41279
I20260810 00:57:08.021227 12535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:41279 every 8 connection(s)
I20260810 00:57:08.022364 12444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:57:08.025704 12536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:08.027340 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 12444
I20260810 00:57:08.027555 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:57:08.028362 10629 external_mini_cluster.cc:1428] Setting key ff393acecc45594d949ac0abdbe3cedf
I20260810 00:57:08.033730 12536 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:57:08 dist-test-slave-jlzn krb5kdc[12424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323427, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:57:08.055646 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.036922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:41279 (local address 127.0.0.1:46402)
0810 00:57:08.037754 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.037805 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:08.040099 (+ 2294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:08.040886 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:08.040892 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:08.041573 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:08.041993 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:08.042014 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:08.043659 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:08.043663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:08.044725 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:08.044737 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:08.045830 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:08.045835 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:08.045957 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:08.046514 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:08.046535 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:08.048563 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:08.052231 (+ 3668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:08.052241 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:08.052254 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:08.052620 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:08.053096 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:08.053099 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:08.053101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:08.053260 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:08.054039 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:08.054044 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:08.054221 (+ 177us) client_negotiation.cc:769] Sending connection context
0810 00:57:08.054402 (+ 181us) client_negotiation.cc:240] Negotiation successful
0810 00:57:08.054723 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":334,"threads_started":1}
I20260810 00:57:08.055691 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.036922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46402 (local address 127.10.97.126:41279)
0810 00:57:08.038653 (+ 1731us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:08.038661 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:08.038706 (+ 45us) server_negotiation.cc:407] Connection header received
0810 00:57:08.040446 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:08.040452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:08.040669 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:08.040818 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:08.042197 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:08.043497 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:08.044861 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:08.045642 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:08.045814 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:08.048782 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:08.048791 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:08.048804 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:08.048840 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:08.052120 (+ 3280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:08.052761 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:08.052769 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:08.052772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:08.052988 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:08.053367 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:08.053372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:08.053374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:08.053925 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:08.054100 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:08.054575 (+ 475us) server_negotiation.cc:299] Negotiation successful
0810 00:57:08.054874 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":1543,"spinlock_wait_cycles":15744,"thread_start_us":1315,"threads_started":1}
I20260810 00:57:08.058321 12536 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:57:08.059221 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.058873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:43875 (local address 127.0.0.1:34778)
0810 00:57:08.059030 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.059119 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 00:57:08.059427 12470 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.97.125:43875: connect: Connection refused (error 111)
I20260810 00:57:08.060863 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.060533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:43875 (local address 127.0.0.1:34794)
0810 00:57:08.060686 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.060749 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 00:57:08.061049 12536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.125:43875: Network error: Client connection negotiation failed: client connection to 127.10.97.125:43875: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.10.97.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.125.keytab.
Entry for principal kudu/127.10.97.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.125.keytab.
I20260810 00:57:08.092200 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.125:43875
--webserver_interface=127.10.97.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.125.keytab
--principal=kudu/127.10.97.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:38199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:41279,127.10.97.125:43875,127.10.97.124:46153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:57:08.110334 12536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } attempt: 1
I20260810 00:57:08.111292 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.110870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:43875 (local address 127.0.0.1:34808)
0810 00:57:08.111048 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.111154 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 00:57:08.112335 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.111996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:43875 (local address 127.0.0.1:34812)
0810 00:57:08.112139 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.112208 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 00:57:08.112507 12536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.125:43875: Network error: Client connection negotiation failed: client connection to 127.10.97.125:43875: connect: Connection refused (error 111)
I20260810 00:57:08.180732 12536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } attempt: 2
I20260810 00:57:08.181903 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.181431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:43875 (local address 127.0.0.1:34828)
0810 00:57:08.181665 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.181764 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 00:57:08.182837 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.182517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:43875 (local address 127.0.0.1:34842)
0810 00:57:08.182636 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.182716 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 00:57:08.183010 12536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.125:43875: Network error: Client connection negotiation failed: client connection to 127.10.97.125:43875: connect: Connection refused (error 111)
I20260810 00:57:08.274230 12536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } attempt: 3
I20260810 00:57:08.275308 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.274871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:43875 (local address 127.0.0.1:34854)
0810 00:57:08.275070 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.275169 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 00:57:08.276444 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.276098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:43875 (local address 127.0.0.1:34864)
0810 00:57:08.276229 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.276305 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 00:57:08.276661 12536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.125:43875: Network error: Client connection negotiation failed: client connection to 127.10.97.125:43875: connect: Connection refused (error 111)
W20260810 00:57:08.407106 12549 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 00:57:08.407395 12549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:08.407495 12549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:08.407603 12549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:08.414419 12549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:57:08.414494 12549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:08.414580 12549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:08.414645 12549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:57:08.414728 12549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:57:08.419874 12536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } attempt: 4
I20260810 00:57:08.420996 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.420500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:43875 (local address 127.0.0.1:34866)
0810 00:57:08.420765 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.420866 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260810 00:57:08.421854 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.421563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:43875 (local address 127.0.0.1:34870)
0810 00:57:08.421672 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.421737 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 00:57:08.422020 12536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.125:43875: Network error: Client connection negotiation failed: client connection to 127.10.97.125:43875: connect: Connection refused (error 111)
I20260810 00:57:08.424409 12549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:38199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:41279,127.10.97.125:43875,127.10.97.124:46153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.125.keytab
--principal=kudu/127.10.97.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.125:43875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.97.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:08.425496 12549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:08.426679 12549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:08.437085 12556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:08.437172 12557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:08.437122 12560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:08.438076 12549 server_base.cc:1034] running on GCE node
I20260810 00:57:08.438751 12549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:08.439957 12549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:08.441179 12549 hybrid_clock.cc:648] HybridClock initialized: now 1786323428441109 us; error 34 us; skew 500 ppm
I20260810 00:57:08.441465 12549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:08 dist-test-slave-jlzn krb5kdc[12424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323428, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:08.444419 12549 init.cc:377] Logged in from keytab as kudu/127.10.97.125@KRBTEST.COM (short username kudu)
I20260810 00:57:08.445701 12549 webserver.cc:460] Webserver started at http://127.10.97.125:45971/ using document root <none> and password file <none>
I20260810 00:57:08.446317 12549 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:08.446416 12549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:08.446707 12549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:08.449482 12549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-1/data/instance:
uuid: "f2921d6d955d488ab2121bc42010b4ed"
format_stamp: "Formatted at 2026-08-10 00:57:08 on dist-test-slave-jlzn"
server_key: "c60bf7e3015760b608915f51658cb323"
server_key_iv: "8d24e0c31b051e5135098cdfbc09621d"
server_key_version: "encryptionkey@0"
I20260810 00:57:08.450238 12549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-1/wal/instance:
uuid: "f2921d6d955d488ab2121bc42010b4ed"
format_stamp: "Formatted at 2026-08-10 00:57:08 on dist-test-slave-jlzn"
server_key: "c60bf7e3015760b608915f51658cb323"
server_key_iv: "8d24e0c31b051e5135098cdfbc09621d"
server_key_version: "encryptionkey@0"
I20260810 00:57:08.454944 12549 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260810 00:57:08.458079 12568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:08.459256 12549 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:57:08.459403 12549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-1/wal
uuid: "f2921d6d955d488ab2121bc42010b4ed"
format_stamp: "Formatted at 2026-08-10 00:57:08 on dist-test-slave-jlzn"
server_key: "c60bf7e3015760b608915f51658cb323"
server_key_iv: "8d24e0c31b051e5135098cdfbc09621d"
server_key_version: "encryptionkey@0"
I20260810 00:57:08.459590 12549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:08.498461 12549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:08.501921 12549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:08.502161 12549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:08.518689 12549 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.125:43875
I20260810 00:57:08.518714 12651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.125:43875 every 8 connection(s)
I20260810 00:57:08.519882 12549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-1/data/info.pb
I20260810 00:57:08.523201 12652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:08.523599 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 12549
I20260810 00:57:08.523762 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-1/wal/instance
I20260810 00:57:08.524180 10629 external_mini_cluster.cc:1428] Setting key ec21ddc92b7d4a9c22bb757b4fa69909
I20260810 00:57:08.529268 12652 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:57:08 dist-test-slave-jlzn krb5kdc[12424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323428, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.125@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:57:08.548177 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.531602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46412 (local address 127.10.97.126:41279)
0810 00:57:08.531831 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:08.531838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:08.532785 (+ 947us) server_negotiation.cc:407] Connection header received
0810 00:57:08.534564 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:08.534570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:08.534657 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:08.534770 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:08.536101 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:08.536972 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:08.538866 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:08.539276 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:08.539368 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:08.542932 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:08.542936 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:08.542940 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:08.542971 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:08.545400 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:08.546052 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:08.546056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:08.546058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:08.546177 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:08.546564 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:08.546568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:08.546570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:08.546840 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:08.546984 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:08.547603 (+ 619us) server_negotiation.cc:299] Negotiation successful
0810 00:57:08.547872 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 00:57:08.548969 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.531928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:41279 (local address 127.0.0.1:46412)
0810 00:57:08.532623 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.532668 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:08.534319 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:08.534925 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:08.534931 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:08.535402 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:08.535946 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:08.535964 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:08.537097 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:08.537100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:08.538726 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:08.538741 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:08.539488 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:08.539494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:08.539736 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:08.540382 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:08.540413 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:08.542731 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:08.545527 (+ 2796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:08.545535 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:08.545548 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:08.545930 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:08.546277 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:08.546282 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:08.546286 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:08.546452 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:08.546961 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:08.546968 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:08.547258 (+ 290us) client_negotiation.cc:769] Sending connection context
0810 00:57:08.547474 (+ 216us) client_negotiation.cc:240] Negotiation successful
0810 00:57:08.547920 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":279,"threads_started":1}
I20260810 00:57:08.550572 12652 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.97.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.124@KRBTEST.COM" created.
Aug 10 00:57:08 dist-test-slave-jlzn krb5kdc[12424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323428, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.125@KRBTEST.COM for kudu/127.10.97.125@KRBTEST.COM
I20260810 00:57:08.564078 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.551268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:43875 (local address 127.0.0.1:34884)
0810 00:57:08.551420 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.551437 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:08.551542 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:08.552687 (+ 1145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:08.552690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:08.553007 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:08.553130 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:08.553139 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:08.554997 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:08.555000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:08.555893 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:08.555905 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:08.556672 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:08.556676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:08.556814 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:08.557244 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:08.557264 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:08.559096 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:08.561879 (+ 2783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:08.561884 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:08.561888 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:08.562214 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:08.562585 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:08.562590 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:08.562594 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:08.562716 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:08.563295 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:08.563300 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:08.563438 (+ 138us) client_negotiation.cc:769] Sending connection context
0810 00:57:08.563574 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 00:57:08.563776 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 00:57:08.564436 12656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.551528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34884 (local address 127.10.97.125:43875)
0810 00:57:08.552238 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:08.552246 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:08.552278 (+ 32us) server_negotiation.cc:407] Connection header received
0810 00:57:08.552331 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:08.552338 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:08.552536 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:08.552683 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:08.553301 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:08.554876 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:08.556004 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:08.556533 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:08.556650 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:08.559286 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:08.559297 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:08.559307 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:08.559334 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:08.561765 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:08.562327 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:08.562332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:08.562334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:08.562476 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:08.562810 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:08.562815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:08.562818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:08.563181 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:08.563358 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:08.563788 (+ 430us) server_negotiation.cc:299] Negotiation successful
0810 00:57:08.564050 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":373,"threads_started":1}
I20260810 00:57:08.566553 12652 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:57:08.567616 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.567167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40488)
0810 00:57:08.567319 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.567423 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 00:57:08.567924 12578 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
I20260810 00:57:08.569502 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.569147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40498)
0810 00:57:08.569297 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.569388 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 00:57:08.569692 12652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:46153: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.124.keytab.
Entry for principal kudu/127.10.97.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.124.keytab.
I20260810 00:57:08.587723 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.124:46153
--webserver_interface=127.10.97.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.124.keytab
--principal=kudu/127.10.97.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:38199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:41279,127.10.97.125:43875,127.10.97.124:46153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:57:08.618984 12652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } attempt: 1
I20260810 00:57:08.620049 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.619571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40506)
0810 00:57:08.619750 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.619883 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 00:57:08.620971 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.620608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40522)
0810 00:57:08.620740 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.620841 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 00:57:08.621196 12652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:46153: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
I20260810 00:57:08.689414 12652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } attempt: 2
I20260810 00:57:08.690551 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.690029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40526)
0810 00:57:08.690254 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.690367 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260810 00:57:08.691566 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.691157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40530)
0810 00:57:08.691306 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.691398 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 00:57:08.691787 12652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:46153: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
I20260810 00:57:08.721244 12536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } attempt: 5
Aug 10 00:57:08 dist-test-slave-jlzn krb5kdc[12424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323427, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for kudu/127.10.97.125@KRBTEST.COM
I20260810 00:57:08.735086 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.721927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:43875 (local address 127.0.0.1:34900)
0810 00:57:08.722141 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.722162 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:08.722276 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:08.722609 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:08.722612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:08.722978 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:08.723164 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:08.723174 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:08.724571 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:08.724574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:08.725433 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:08.725444 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:08.726428 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:08.726432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:08.726598 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:08.727174 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:08.727194 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:08.729411 (+ 2217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:08.732971 (+ 3560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:08.732975 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:08.732978 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:08.733321 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:08.733716 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:08.733719 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:08.733721 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:08.733839 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:08.734423 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:08.734426 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:08.734553 (+ 127us) client_negotiation.cc:769] Sending connection context
0810 00:57:08.734654 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 00:57:08.734847 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 00:57:08.735313 12656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.722077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34900 (local address 127.10.97.125:43875)
0810 00:57:08.722269 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:08.722275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:08.722299 (+ 24us) server_negotiation.cc:407] Connection header received
0810 00:57:08.722365 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:08.722370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:08.722465 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:08.722586 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:08.723320 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:08.724448 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:08.725601 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:08.726289 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:08.726429 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:08.729613 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:08.729619 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:08.729624 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:08.729664 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:08.732844 (+ 3180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:08.733435 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:08.733441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:08.733444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:08.733616 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:08.733935 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:08.733940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:08.733943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:08.734321 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:08.734480 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:08.734781 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 00:57:08.735017 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 00:57:08.736272 12536 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:57:08.737214 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.736870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40542)
0810 00:57:08.737013 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.737083 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 00:57:08.738104 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.737823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40554)
0810 00:57:08.737942 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.738002 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 00:57:08.738310 12536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:46153: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
I20260810 00:57:08.783059 12652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } attempt: 3
I20260810 00:57:08.784216 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.783656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40564)
0810 00:57:08.783904 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.784023 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 00:57:08.785301 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.784808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40568)
0810 00:57:08.785054 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.785165 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260810 00:57:08.785538 12652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:46153: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
I20260810 00:57:08.789494 12536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } attempt: 1
I20260810 00:57:08.790407 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.790034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40582)
0810 00:57:08.790187 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.790272 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 00:57:08.791291 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.790986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40588)
0810 00:57:08.791119 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.791187 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 00:57:08.791517 12536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:46153: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
I20260810 00:57:08.859704 12536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } attempt: 2
I20260810 00:57:08.860836 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.860385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40598)
0810 00:57:08.860566 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.860676 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 00:57:08.861857 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.861521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40614)
0810 00:57:08.861679 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.861756 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 00:57:08.862067 12536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:46153: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
W20260810 00:57:08.914460 12666 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 00:57:08.914783 12666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:08.914883 12666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:08.914989 12666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:08.921878 12666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:57:08.921958 12666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:08.922047 12666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:08.922112 12666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:57:08.922210 12666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:57:08.928768 12652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } attempt: 4
I20260810 00:57:08.929917 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.929382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40622)
0810 00:57:08.929608 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.929710 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 00:57:08.930874 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.930522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40634)
0810 00:57:08.930662 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.930746 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 00:57:08.931067 12652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:46153: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
I20260810 00:57:08.932327 12666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:38199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:41279,127.10.97.125:43875,127.10.97.124:46153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.124.keytab
--principal=kudu/127.10.97.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.124:46153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.97.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:08.933455 12666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:08.934671 12666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:08.945338 12675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:08.945534 12679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:08.945775 12674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:08.946437 12666 server_base.cc:1034] running on GCE node
I20260810 00:57:08.947311 12666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:08.948792 12666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:08.950008 12666 hybrid_clock.cc:648] HybridClock initialized: now 1786323428949948 us; error 29 us; skew 500 ppm
I20260810 00:57:08.950321 12666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:08 dist-test-slave-jlzn krb5kdc[12424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323428, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:08.953466 12666 init.cc:377] Logged in from keytab as kudu/127.10.97.124@KRBTEST.COM (short username kudu)
I20260810 00:57:08.955166 12666 webserver.cc:460] Webserver started at http://127.10.97.124:40591/ using document root <none> and password file <none>
I20260810 00:57:08.955760 12666 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:08.955886 12666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:08.956175 12666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:08.958853 12666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-2/data/instance:
uuid: "717c4ed8dad2487e8df79637134e0131"
format_stamp: "Formatted at 2026-08-10 00:57:08 on dist-test-slave-jlzn"
server_key: "db781c7b1448b0b4a345de232dc15458"
server_key_iv: "3b02670745c67ef1d1fe93a20ab6d7ce"
server_key_version: "encryptionkey@0"
I20260810 00:57:08.959625 12666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-2/wal/instance:
uuid: "717c4ed8dad2487e8df79637134e0131"
format_stamp: "Formatted at 2026-08-10 00:57:08 on dist-test-slave-jlzn"
server_key: "db781c7b1448b0b4a345de232dc15458"
server_key_iv: "3b02670745c67ef1d1fe93a20ab6d7ce"
server_key_version: "encryptionkey@0"
I20260810 00:57:08.964789 12666 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260810 00:57:08.967968 12689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:08.968266 12536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } attempt: 3
I20260810 00:57:08.969318 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.968810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40636)
0810 00:57:08.969033 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.969131 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 00:57:08.969347 12666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 00:57:08.969489 12666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-2/wal
uuid: "717c4ed8dad2487e8df79637134e0131"
format_stamp: "Formatted at 2026-08-10 00:57:08 on dist-test-slave-jlzn"
server_key: "db781c7b1448b0b4a345de232dc15458"
server_key_iv: "3b02670745c67ef1d1fe93a20ab6d7ce"
server_key_version: "encryptionkey@0"
I20260810 00:57:08.969702 12666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:08.970340 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:08.969974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40648)
0810 00:57:08.970167 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:08.970235 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260810 00:57:08.970541 12536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:46153: Network error: Client connection negotiation failed: client connection to 127.10.97.124:46153: connect: Connection refused (error 111)
I20260810 00:57:08.983471 12666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:08.987272 12666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:08.987515 12666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:09.003551 12666 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.124:46153
I20260810 00:57:09.003568 12759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.124:46153 every 8 connection(s)
I20260810 00:57:09.004757 12666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-2/data/info.pb
I20260810 00:57:09.007990 12762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:09.013844 12762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:57:09.014472 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 12666
I20260810 00:57:09.014658 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786323077328248-10629-0/minicluster-data/master-2/wal/instance
I20260810 00:57:09.015085 10629 external_mini_cluster.cc:1428] Setting key f15236513e629a9e896ff40907eb7e72
Aug 10 00:57:09 dist-test-slave-jlzn krb5kdc[12424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.124@KRBTEST.COM
Aug 10 00:57:09 dist-test-slave-jlzn krb5kdc[12424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323428, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.124@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:57:09.034297 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:09.016125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46424 (local address 127.10.97.126:41279)
0810 00:57:09.016313 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.016319 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.017621 (+ 1302us) server_negotiation.cc:407] Connection header received
0810 00:57:09.019289 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.019295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.019361 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.019455 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.020875 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.021926 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.023989 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.024559 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.024672 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:09.028652 (+ 3980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:09.028657 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:09.028660 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:09.028689 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:09.031285 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.032133 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.032137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.032140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.032323 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.032794 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.032799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.032802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.033084 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:09.033229 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:09.033727 (+ 498us) server_negotiation.cc:299] Negotiation successful
0810 00:57:09.033949 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 00:57:09.035061 12765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:09.016651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:41279 (local address 127.0.0.1:46424)
0810 00:57:09.017479 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:09.017514 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:09.019067 (+ 1553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:09.019551 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:09.019558 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:09.020246 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:09.020662 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:09.020690 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.022121 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.022136 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:09.023833 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:09.023848 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.024754 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.024800 (+ 46us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:09.024966 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:09.025554 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:09.025581 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:09.028464 (+ 2883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:09.031411 (+ 2947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:09.031422 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:09.031432 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:09.032004 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:09.032437 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:09.032440 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:09.032443 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:09.032659 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:09.033227 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:09.033234 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:09.033394 (+ 160us) client_negotiation.cc:769] Sending connection context
0810 00:57:09.033594 (+ 200us) client_negotiation.cc:240] Negotiation successful
0810 00:57:09.034008 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":44,"thread_start_us":343,"threads_started":1}
I20260810 00:57:09.035079 12768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:09.016651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40652 (local address 127.10.97.124:46153)
0810 00:57:09.017536 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.017544 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.017585 (+ 41us) server_negotiation.cc:407] Connection header received
0810 00:57:09.019058 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.019078 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.019287 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.019437 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.020873 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.021977 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.023204 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.023968 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.024140 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:09.028347 (+ 4207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:09.028357 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:09.028371 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:09.028406 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:09.031457 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.032254 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.032262 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.032265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.032446 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.032804 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.032809 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.032812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.033243 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:09.033536 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:09.033834 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 00:57:09.034258 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":124,"server-negotiator.queue_time_us":693,"spinlock_wait_cycles":11136,"thread_start_us":501,"threads_started":1}
I20260810 00:57:09.036513 12762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:57:09 dist-test-slave-jlzn krb5kdc[12424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323428, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.124@KRBTEST.COM for kudu/127.10.97.125@KRBTEST.COM
Aug 10 00:57:09 dist-test-slave-jlzn krb5kdc[12424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:57:09.049595 12765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:09.037068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:43875 (local address 127.0.0.1:34912)
0810 00:57:09.037222 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:09.037238 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:09.037329 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:09.037625 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:09.037628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:09.037929 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:09.038095 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:09.038107 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.039378 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.039382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:09.040593 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:09.040607 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.041425 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.041429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:09.041573 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:09.042000 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:09.042015 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:09.044277 (+ 2262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:09.047383 (+ 3106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:09.047387 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:09.047389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:09.047715 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:09.048146 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:09.048149 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:09.048151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:09.048286 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:09.048779 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:09.048783 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:09.048910 (+ 127us) client_negotiation.cc:769] Sending connection context
0810 00:57:09.049020 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 00:57:09.049264 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 00:57:09.049688 12656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:09.037163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34912 (local address 127.10.97.125:43875)
0810 00:57:09.037319 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.037325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.037342 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:57:09.037427 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.037433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.037508 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.037612 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.038259 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.039246 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.040723 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.041299 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.041423 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:09.044459 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:09.044464 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:09.044467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:09.044497 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:09.047262 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.047867 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.047872 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.047875 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.048040 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.048396 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.048400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.048402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.048659 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:09.048796 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:09.049130 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 00:57:09.049327 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 00:57:09.050534 12762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:57:09.052119 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:09.038005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46440 (local address 127.10.97.126:41279)
0810 00:57:09.038155 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.038160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.038176 (+ 16us) server_negotiation.cc:407] Connection header received
0810 00:57:09.038280 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.038283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.038333 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.038397 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.039734 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.040800 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.042119 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.042728 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.042852 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:09.046881 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:09.046887 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:09.046892 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:09.046929 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:09.049515 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.050159 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.050165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.050168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.050301 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.050774 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.050779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.050783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.051071 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:09.051334 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:09.051593 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 00:57:09.051793 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 00:57:09 dist-test-slave-jlzn krb5kdc[12424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323428, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.124@KRBTEST.COM for kudu/127.10.97.124@KRBTEST.COM
Aug 10 00:57:09 dist-test-slave-jlzn krb5kdc[12424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.125@KRBTEST.COM
I20260810 00:57:09.062466 12765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:09.051219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40668)
0810 00:57:09.051407 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:09.051419 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:09.051519 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:09.051771 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:09.051775 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:09.052245 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:09.052420 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:09.052432 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.053841 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.053845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:09.054625 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:09.054638 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.055229 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.055235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:09.055416 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:09.055895 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:09.055910 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:09.057773 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:09.060354 (+ 2581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:09.060358 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:09.060360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:09.060689 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:09.061058 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:09.061063 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:09.061066 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:09.061239 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:09.061686 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:09.061690 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:09.061801 (+ 111us) client_negotiation.cc:769] Sending connection context
0810 00:57:09.061921 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 00:57:09.062094 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260810 00:57:09.062650 12768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:09.051163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40668 (local address 127.10.97.124:46153)
0810 00:57:09.051311 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.051315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.051477 (+ 162us) server_negotiation.cc:407] Connection header received
0810 00:57:09.051608 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.051613 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.051674 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.051740 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.052939 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.053729 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.054748 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.055102 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.055208 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:09.057927 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:09.057932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:09.057935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:09.057969 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:09.060231 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.060818 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.060822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.060824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.060949 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.061341 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.061344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.061346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.061569 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:09.061698 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:09.062042 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 00:57:09.062270 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 00:57:09.066753 12656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:09.053498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34926 (local address 127.10.97.125:43875)
0810 00:57:09.053647 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.053652 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.053668 (+ 16us) server_negotiation.cc:407] Connection header received
0810 00:57:09.053813 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.053819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.053876 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.053947 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.055029 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.055773 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.056991 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.057437 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.057528 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:09.061695 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:09.061701 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:09.061705 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:09.061739 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:09.064471 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.065032 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.065038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.065042 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.065174 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.065508 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.065513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.065517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.065769 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:09.066010 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:09.066270 (+ 260us) server_negotiation.cc:299] Negotiation successful
0810 00:57:09.066454 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 00:57:09.066926 12762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131: Bootstrap starting.
I20260810 00:57:09.070330 12762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:09.070883 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:57:09.071401 12762 log.cc:824] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:09.074157 12762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131: No bootstrap required, opened a new log
I20260810 00:57:09.073478 12502 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ad5dbcb943354bc38c8e62e867ce513f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46440
I20260810 00:57:09.084349 12762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5dbcb943354bc38c8e62e867ce513f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } } peers { permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } } peers { permanent_uuid: "717c4ed8dad2487e8df79637134e0131" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } }
I20260810 00:57:09.084708 12762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 717c4ed8dad2487e8df79637134e0131, State: Initialized, Role: FOLLOWER
I20260810 00:57:09.085155 12762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5dbcb943354bc38c8e62e867ce513f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } } peers { permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } } peers { permanent_uuid: "717c4ed8dad2487e8df79637134e0131" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } }
I20260810 00:57:09.085973 12773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5dbcb943354bc38c8e62e867ce513f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } } peers { permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } } peers { permanent_uuid: "717c4ed8dad2487e8df79637134e0131" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } } }
I20260810 00:57:09.086274 12773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:57:09.087066 12762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 00:57:09.095655 12787 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 00:57:09.095774 12787 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 00:57:09.093818 12666 master_runner.cc:186] Error getting master registration for 127.10.97.126:41279: OK, instance_id { permanent_uuid: "ad5dbcb943354bc38c8e62e867ce513f" instance_seqno: 1786323428016580 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 00:57:09.104246 12666 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 00:57:09.147734 12536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } attempt: 4
Aug 10 00:57:09 dist-test-slave-jlzn krb5kdc[12424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323427, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for kudu/127.10.97.124@KRBTEST.COM
I20260810 00:57:09.163864 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:09.148363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40678)
0810 00:57:09.148537 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:09.148556 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:09.148668 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:09.148994 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:09.148999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:09.149488 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:09.149691 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:09.149703 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.151108 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.151113 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:09.152690 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:09.152708 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.153694 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.153699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:09.153885 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:09.154524 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:09.154556 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:09.157395 (+ 2839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:09.161163 (+ 3768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:09.161169 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:09.161173 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:09.161661 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:09.162018 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:09.162023 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:09.162027 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:09.162220 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:09.162906 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:09.162910 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:09.163065 (+ 155us) client_negotiation.cc:769] Sending connection context
0810 00:57:09.163190 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 00:57:09.163445 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 00:57:09.163864 12768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:09.148441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40678 (local address 127.10.97.124:46153)
0810 00:57:09.148629 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.148635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.148657 (+ 22us) server_negotiation.cc:407] Connection header received
0810 00:57:09.148784 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.148790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.148862 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.148975 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.149841 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.150981 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.152838 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.153565 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.153688 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:09.157589 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:09.157597 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:09.157603 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:09.157643 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:09.161023 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.161768 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.161774 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.161778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.161917 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.162328 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.162333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.162336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.162769 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:09.162968 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:09.163304 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 00:57:09.163504 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 00:57:09.169802 12536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f: Bootstrap starting.
I20260810 00:57:09.173727 12536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:09.175009 12536 log.cc:824] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:09.178107 12536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f: No bootstrap required, opened a new log
I20260810 00:57:09.185465 12536 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5dbcb943354bc38c8e62e867ce513f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } } peers { permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } } peers { permanent_uuid: "717c4ed8dad2487e8df79637134e0131" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } }
I20260810 00:57:09.185845 12536 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5dbcb943354bc38c8e62e867ce513f, State: Initialized, Role: FOLLOWER
I20260810 00:57:09.186354 12536 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5dbcb943354bc38c8e62e867ce513f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } } peers { permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } } peers { permanent_uuid: "717c4ed8dad2487e8df79637134e0131" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } }
I20260810 00:57:09.187650 12793 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5dbcb943354bc38c8e62e867ce513f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } } peers { permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } } peers { permanent_uuid: "717c4ed8dad2487e8df79637134e0131" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } } }
I20260810 00:57:09.188133 12793 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:57:09.188732 12536 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:57:09.188922 12502 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 00:57:09.189141 12502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:09.191606 12502 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5dbcb943354bc38c8e62e867ce513f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } } peers { permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } } peers { permanent_uuid: "717c4ed8dad2487e8df79637134e0131" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } }
I20260810 00:57:09.193509 12722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad5dbcb943354bc38c8e62e867ce513f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "717c4ed8dad2487e8df79637134e0131"
I20260810 00:57:09.193959 12722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:09.197316 12722 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5dbcb943354bc38c8e62e867ce513f in term 1.
I20260810 00:57:09.197896 12502 leader_election.cc:288] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [CANDIDATE]: Term 1 election: Requested vote from peers f2921d6d955d488ab2121bc42010b4ed (127.10.97.125:43875), 717c4ed8dad2487e8df79637134e0131 (127.10.97.124:46153)
I20260810 00:57:09.197935 12471 leader_election.cc:302] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 717c4ed8dad2487e8df79637134e0131, ad5dbcb943354bc38c8e62e867ce513f; no voters:
I20260810 00:57:09.198331 12793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:09.209319 12793 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [term 1 LEADER]: Becoming Leader. State: Replica: ad5dbcb943354bc38c8e62e867ce513f, State: Running, Role: LEADER
I20260810 00:57:09.210309 12793 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5dbcb943354bc38c8e62e867ce513f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } } peers { permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } } peers { permanent_uuid: "717c4ed8dad2487e8df79637134e0131" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } }
I20260810 00:57:09.224865 12798 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad5dbcb943354bc38c8e62e867ce513f. Latest consensus state: current_term: 1 leader_uuid: "ad5dbcb943354bc38c8e62e867ce513f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5dbcb943354bc38c8e62e867ce513f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } } peers { permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } } peers { permanent_uuid: "717c4ed8dad2487e8df79637134e0131" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } } }
I20260810 00:57:09.225215 12798 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:09.226271 12820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:57:09.230247 12652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } attempt: 5
W20260810 00:57:09.239352 12695 tablet.cc:2369] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:57:09.243503 12820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:57:09.249646 12656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:09.222836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34928 (local address 127.10.97.125:43875)
0810 00:57:09.223020 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.223026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.232762 (+ 9736us) server_negotiation.cc:407] Connection header received
0810 00:57:09.233362 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.233368 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.233446 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.233520 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.235647 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.236820 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.238819 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.239432 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.239528 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:09.242710 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:09.242717 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:09.242721 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:09.242759 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:09.246264 (+ 3505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.247175 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.247181 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.247184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.247364 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.247826 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.247831 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.247834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.248207 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:09.248424 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:09.249053 (+ 629us) server_negotiation.cc:299] Negotiation successful
0810 00:57:09.249290 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260810 00:57:09.256845 12475 tablet.cc:2369] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:57:09.259454 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:09.223219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46442 (local address 127.10.97.126:41279)
0810 00:57:09.223380 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.223385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.232403 (+ 9018us) server_negotiation.cc:407] Connection header received
0810 00:57:09.232696 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.232702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.232782 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.232848 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.244359 (+ 11511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.245446 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.247268 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.247966 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.248080 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:09.251017 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:09.251023 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:09.251028 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:09.251071 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:09.255694 (+ 4623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.256564 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.256569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.256573 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.256768 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.257205 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.257210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.257213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.257575 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:09.258687 (+ 1112us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:09.258831 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 00:57:09.259073 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 00:57:09 dist-test-slave-jlzn krb5kdc[12424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323428, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.125@KRBTEST.COM for kudu/127.10.97.124@KRBTEST.COM
I20260810 00:57:09.270517 12768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:09.226112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40682 (local address 127.10.97.124:46153)
0810 00:57:09.226278 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.226284 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.252011 (+ 25727us) server_negotiation.cc:407] Connection header received
0810 00:57:09.252266 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.252272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.252370 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.252436 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.253922 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.255138 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.256869 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.257445 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.257565 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:09.260823 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:09.260830 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:09.260835 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:09.260877 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:09.264942 (+ 4065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.265774 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.265780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.265784 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.266048 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.266793 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.266798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.266801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.267377 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:09.269301 (+ 1924us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:09.269511 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 00:57:09.269914 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":67}
I20260810 00:57:09.274168 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:09.230744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:46153 (local address 127.0.0.1:40686)
0810 00:57:09.231068 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:09.231087 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:09.231208 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:09.253525 (+ 22317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:09.253530 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:09.253964 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:09.254188 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:09.254200 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.256059 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.256064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:09.257322 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:09.257337 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.258401 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.258407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:09.258621 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:09.259143 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:09.259166 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:09.261720 (+ 2554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:09.270492 (+ 8772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:09.270497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:09.270501 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:09.270999 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:09.271851 (+ 852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:09.271857 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:09.271861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:09.272029 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:09.273207 (+ 1178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:09.273212 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:09.273355 (+ 143us) client_negotiation.cc:769] Sending connection context
0810 00:57:09.273494 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 00:57:09.273763 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209}
I20260810 00:57:09.276160 12830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:09.231394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40686 (local address 127.10.97.124:46153)
0810 00:57:09.251368 (+ 19974us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.251374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.251395 (+ 21us) server_negotiation.cc:407] Connection header received
0810 00:57:09.251447 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.251452 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.251544 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.254389 (+ 2845us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.254678 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.255900 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.257501 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:09.258237 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:09.262085 (+ 3848us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:09.262364 (+ 279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:09.262371 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:09.262376 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:09.262410 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:09.269122 (+ 6712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.271198 (+ 2076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.271203 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.271206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.271360 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:09.272203 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:09.272208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:09.272212 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:09.272560 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:09.275400 (+ 2840us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:09.275515 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 00:57:09.275706 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":19852,"thread_start_us":20247,"threads_started":1}
I20260810 00:57:09.281471 12652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed: Bootstrap starting.
I20260810 00:57:09.284868 12722 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131 [term 1 FOLLOWER]: Refusing update from remote peer ad5dbcb943354bc38c8e62e867ce513f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:09.285871 12798 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [LEADER]: Connected to new peer: Peer: permanent_uuid: "717c4ed8dad2487e8df79637134e0131" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:57:09.287268 12652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:09.288667 12652 log.cc:824] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:09.302834 12652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed: No bootstrap required, opened a new log
I20260810 00:57:09.317225 12773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad5dbcb943354bc38c8e62e867ce513f. Latest consensus state: current_term: 1 leader_uuid: "ad5dbcb943354bc38c8e62e867ce513f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5dbcb943354bc38c8e62e867ce513f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } } peers { permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } } peers { permanent_uuid: "717c4ed8dad2487e8df79637134e0131" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } } }
I20260810 00:57:09.317517 12773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:57:09.322589 12652 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5dbcb943354bc38c8e62e867ce513f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } } peers { permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } } peers { permanent_uuid: "717c4ed8dad2487e8df79637134e0131" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } }
I20260810 00:57:09.323004 12652 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2921d6d955d488ab2121bc42010b4ed, State: Initialized, Role: FOLLOWER
I20260810 00:57:09.323501 12652 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5dbcb943354bc38c8e62e867ce513f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } } peers { permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } } peers { permanent_uuid: "717c4ed8dad2487e8df79637134e0131" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } }
I20260810 00:57:09.329236 12773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad5dbcb943354bc38c8e62e867ce513f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5dbcb943354bc38c8e62e867ce513f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } } peers { permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } } peers { permanent_uuid: "717c4ed8dad2487e8df79637134e0131" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } } }
I20260810 00:57:09.329208 12847 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5dbcb943354bc38c8e62e867ce513f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } } peers { permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } } peers { permanent_uuid: "717c4ed8dad2487e8df79637134e0131" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } } }
I20260810 00:57:09.329463 12773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:57:09.329557 12847 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:57:09.336818 12798 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad5dbcb943354bc38c8e62e867ce513f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5dbcb943354bc38c8e62e867ce513f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } } peers { permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } } peers { permanent_uuid: "717c4ed8dad2487e8df79637134e0131" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } } }
I20260810 00:57:09.337095 12798 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:09.337401 12652 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:57:09.337468 12614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad5dbcb943354bc38c8e62e867ce513f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f2921d6d955d488ab2121bc42010b4ed"
I20260810 00:57:09.337924 12614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:09.338478 12820 catalog_manager.cc:1347] Generated new cluster ID: 2ef1f15a8e6048d9bbbeded6622afb7f
I20260810 00:57:09.338582 12820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:57:09.342067 12614 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5dbcb943354bc38c8e62e867ce513f in term 1.
I20260810 00:57:09.342828 12613 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed [term 1 FOLLOWER]: Refusing update from remote peer ad5dbcb943354bc38c8e62e867ce513f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:09.347941 12793 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:57:09.362445 12860 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 00:57:09.366335 12444 master_runner.cc:186] Error getting master registration for 127.10.97.125:43875: OK, instance_id { permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" instance_seqno: 1786323428513972 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 00:57:09.377321 12444 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 00:57:09.381568 12874 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad5dbcb943354bc38c8e62e867ce513f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5dbcb943354bc38c8e62e867ce513f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } } peers { permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } } peers { permanent_uuid: "717c4ed8dad2487e8df79637134e0131" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } } }
I20260810 00:57:09.382309 12847 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad5dbcb943354bc38c8e62e867ce513f. Latest consensus state: current_term: 1 leader_uuid: "ad5dbcb943354bc38c8e62e867ce513f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5dbcb943354bc38c8e62e867ce513f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } } peers { permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } } peers { permanent_uuid: "717c4ed8dad2487e8df79637134e0131" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } } }
I20260810 00:57:09.382529 12874 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:57:09.382568 12847 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:57:09.400642 12876 catalog_manager.cc:1259] Loaded cluster ID: 2ef1f15a8e6048d9bbbeded6622afb7f
I20260810 00:57:09.400730 12876 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed: loading cluster ID for follower catalog manager: success
W20260810 00:57:09.401818 12876 catalog_manager.cc:1576] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260810 00:57:09.401943 12876 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260810 00:57:09.427574 12793 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad5dbcb943354bc38c8e62e867ce513f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5dbcb943354bc38c8e62e867ce513f" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 41279 } } peers { permanent_uuid: "f2921d6d955d488ab2121bc42010b4ed" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 43875 } } peers { permanent_uuid: "717c4ed8dad2487e8df79637134e0131" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 46153 } } }
I20260810 00:57:09.427902 12793 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:09.433779 12820 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:57:09.435434 12820 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:57:09.449723 12820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad5dbcb943354bc38c8e62e867ce513f: Generated new TSK 0
I20260810 00:57:09.450593 12820 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 00:57:09.564694 10629 mini_kdc.cc:348] Time spent kdestroy: real 0.108s user 0.000s sys 0.029s
W20260810 00:57:09.629087 12541 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:09.605898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46456 (local address 127.10.97.126:41279)
0810 00:57:09.606136 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.606144 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.626915 (+ 20771us) server_negotiation.cc:407] Connection header received
0810 00:57:09.627409 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.627416 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.627507 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.627593 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.628841 (+ 1248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46456: BlockingRecv error: recv got EOF from 127.0.0.1:46456 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260810 00:57:09.631857 12830 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:09.621562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40690 (local address 127.10.97.124:46153)
0810 00:57:09.621758 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.621764 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.630114 (+ 8350us) server_negotiation.cc:407] Connection header received
0810 00:57:09.630371 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.630377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.630461 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.630537 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.631614 (+ 1077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40690: BlockingRecv error: recv got EOF from 127.0.0.1:40690 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260810 00:57:09.633658 12656 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:09.612337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34936 (local address 127.10.97.125:43875)
0810 00:57:09.612512 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.612519 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.629767 (+ 17248us) server_negotiation.cc:407] Connection header received
0810 00:57:09.632202 (+ 2435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.632209 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.632306 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.632383 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.633439 (+ 1056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34936: BlockingRecv error: recv got EOF from 127.0.0.1:34936 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260810 00:57:09.667858 12656 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:09.665586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34942 (local address 127.10.97.125:43875)
0810 00:57:09.665757 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.665763 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.666248 (+ 485us) server_negotiation.cc:407] Connection header received
0810 00:57:09.666461 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.666467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.666595 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.666716 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.667529 (+ 813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34942: BlockingRecv error: recv got EOF from 127.0.0.1:34942 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 00:57:09.682773 12656 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:09.677790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34944 (local address 127.10.97.125:43875)
0810 00:57:09.677941 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.677947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.681403 (+ 3456us) server_negotiation.cc:407] Connection header received
0810 00:57:09.681580 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.681586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.681668 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.681779 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.682539 (+ 760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34944: BlockingRecv error: recv got EOF from 127.0.0.1:34944 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260810 00:57:09.699358 12656 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:09.697476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34950 (local address 127.10.97.125:43875)
0810 00:57:09.697622 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.697627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.698084 (+ 457us) server_negotiation.cc:407] Connection header received
0810 00:57:09.698215 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.698221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.698323 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.698431 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.699124 (+ 693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34950: BlockingRecv error: recv got EOF from 127.0.0.1:34950 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 00:57:09.708272 12656 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:09.706238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34954 (local address 127.10.97.125:43875)
0810 00:57:09.706438 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.706444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.706912 (+ 468us) server_negotiation.cc:407] Connection header received
0810 00:57:09.707085 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.707091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.707168 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.707274 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.708058 (+ 784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34954: BlockingRecv error: recv got EOF from 127.0.0.1:34954 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
W20260810 00:57:09.723363 12656 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:09.721474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34960 (local address 127.10.97.125:43875)
0810 00:57:09.721639 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.721644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.722082 (+ 438us) server_negotiation.cc:407] Connection header received
0810 00:57:09.722232 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.722237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.722316 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.722432 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.723143 (+ 711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34960: BlockingRecv error: recv got EOF from 127.0.0.1:34960 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260810 00:57:09.744552 12656 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:09.742467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34964 (local address 127.10.97.125:43875)
0810 00:57:09.742631 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.742640 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.743120 (+ 480us) server_negotiation.cc:407] Connection header received
0810 00:57:09.743313 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.743319 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.743401 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.743508 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.744318 (+ 810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34964: BlockingRecv error: recv got EOF from 127.0.0.1:34964 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260810 00:57:09.785396 12656 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:09.782882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34970 (local address 127.10.97.125:43875)
0810 00:57:09.783067 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.783073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.783628 (+ 555us) server_negotiation.cc:407] Connection header received
0810 00:57:09.783862 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.783869 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.783962 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.784064 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.785102 (+ 1038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34970: BlockingRecv error: recv got EOF from 127.0.0.1:34970 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260810 00:57:09.854844 12656 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:09.852831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34972 (local address 127.10.97.125:43875)
0810 00:57:09.853061 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.853067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.853395 (+ 328us) server_negotiation.cc:407] Connection header received
0810 00:57:09.853593 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.853597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.853688 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.853781 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.854620 (+ 839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34972: BlockingRecv error: recv got EOF from 127.0.0.1:34972 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260810 00:57:09.987538 12656 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:09.985899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34978 (local address 127.10.97.125:43875)
0810 00:57:09.986117 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:09.986123 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:09.986362 (+ 239us) server_negotiation.cc:407] Connection header received
0810 00:57:09.986518 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:09.986522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:09.986591 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:09.986673 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:09.987297 (+ 624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34978: BlockingRecv error: recv got EOF from 127.0.0.1:34978 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 00:57:10.097582 12787 catalog_manager.cc:1259] Loaded cluster ID: 2ef1f15a8e6048d9bbbeded6622afb7f
I20260810 00:57:10.097679 12787 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131: loading cluster ID for follower catalog manager: success
I20260810 00:57:10.100137 12787 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131: acquiring CA information for follower catalog manager: success
I20260810 00:57:10.101794 12787 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 717c4ed8dad2487e8df79637134e0131: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 00:57:10.250712 12656 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:10.249165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34994 (local address 127.10.97.125:43875)
0810 00:57:10.249386 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:10.249393 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:10.249520 (+ 127us) server_negotiation.cc:407] Connection header received
0810 00:57:10.249698 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:10.249702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:10.249774 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:10.249859 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:10.250494 (+ 635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34994: BlockingRecv error: recv got EOF from 127.0.0.1:34994 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 00:57:10.405265 12876 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed: acquiring CA information for follower catalog manager: success
I20260810 00:57:10.407014 12876 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f2921d6d955d488ab2121bc42010b4ed: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 00:57:10.772568 12992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:10.767182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35002 (local address 127.10.97.125:43875)
0810 00:57:10.767672 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:10.767677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:10.767792 (+ 115us) server_negotiation.cc:407] Connection header received
0810 00:57:10.767996 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:10.768000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:10.768161 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:10.768268 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:10.768688 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:10.769380 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:10.770632 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:10.771008 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:10.771093 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:10.771519 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:10.772069 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:10.772191 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:10.772237 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 00:57:10.772325 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":271,"threads_started":1}
I20260810 00:57:10.783751 12999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:10.778455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40694 (local address 127.10.97.124:46153)
0810 00:57:10.778905 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:10.778911 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:10.778968 (+ 57us) server_negotiation.cc:407] Connection header received
0810 00:57:10.779193 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:10.779197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:10.779336 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:10.779472 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:10.779848 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:10.780522 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:10.781771 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:10.782169 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:10.782274 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:10.782762 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:10.783178 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:10.783300 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:10.783413 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 00:57:10.783492 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":220,"threads_started":1}
I20260810 00:57:10.800158 13008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:10.789712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46472 (local address 127.10.97.126:41279)
0810 00:57:10.793328 (+ 3616us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:10.793335 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:10.794013 (+ 678us) server_negotiation.cc:407] Connection header received
0810 00:57:10.794136 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:10.794142 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:10.794343 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:10.794548 (+ 205us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:10.795141 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:10.795873 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:10.797904 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:10.798410 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:10.798532 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:10.798969 (+ 437us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:10.799248 (+ 279us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:10.799343 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:10.799740 (+ 397us) server_negotiation.cc:299] Negotiation successful
0810 00:57:10.799855 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3481,"thread_start_us":3369,"threads_started":1}
I20260810 00:57:10.800293 12992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:10.789732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35004 (local address 127.10.97.125:43875)
0810 00:57:10.789877 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:10.789883 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:10.794096 (+ 4213us) server_negotiation.cc:407] Connection header received
0810 00:57:10.794248 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:10.794253 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:10.794320 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:10.794392 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:10.795011 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:10.796116 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:10.798175 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:10.798774 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:10.798879 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:10.799524 (+ 645us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:10.799649 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:10.799735 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:10.799964 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 00:57:10.800039 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 00:57:10.802955 12999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:10.789918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40696 (local address 127.10.97.124:46153)
0810 00:57:10.793291 (+ 3373us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:10.793297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:10.796898 (+ 3601us) server_negotiation.cc:407] Connection header received
0810 00:57:10.797040 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:10.797045 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:10.797118 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:10.797197 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:10.797839 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:10.798764 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:10.801377 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:10.801783 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:10.801910 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:10.802295 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:10.802411 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:10.802499 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:10.802667 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 00:57:10.802724 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3257}
I20260810 00:57:10.805187 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 12444
I20260810 00:57:10.806222 12529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:10.921788 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 12444
W20260810 00:57:10.949069 12693 connection.cc:535] server connection from 127.0.0.1:40678 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 00:57:10.949904 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 12549
I20260810 00:57:10.951045 12645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:11.069700 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 12549
I20260810 00:57:11.095261 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 12666
I20260810 00:57:11.096381 12753 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:11.200524 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 12666
2026-08-10T00:57:11Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5059 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 00:57:11 dist-test-slave-jlzn krb5kdc[13019](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 00:57:11 dist-test-slave-jlzn krb5kdc[13019](info): set up 2 sockets
Aug 10 00:57:11 dist-test-slave-jlzn krb5kdc[13019](info): commencing operation
krb5kdc: starting...
W20260810 00:57:12.409349 10629 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/test-user.keytab.
Aug 10 00:57:12 dist-test-slave-jlzn krb5kdc[13019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T00:57:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T00:57:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.97.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
Entry for principal kudu/127.10.97.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab.
I20260810 00:57:12.668740 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:33355
--webserver_interface=127.10.97.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:33975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:33355,127.10.97.125:40863,127.10.97.124:39411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 00:57:12.990528 13041 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 00:57:12.990803 13041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:12.990881 13041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:12.990958 13041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:12.998286 13041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:57:12.998363 13041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:12.998420 13041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:12.998469 13041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:57:12.998533 13041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:57:13.007651 13041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:33355,127.10.97.125:40863,127.10.97.124:39411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.126.keytab
--principal=kudu/127.10.97.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.126:33355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.97.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:13.008735 13041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:13.009907 13041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:13.020867 13049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:13.020838 13050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:13.020795 13053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:13.021672 13041 server_base.cc:1034] running on GCE node
I20260810 00:57:13.022372 13041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:13.023557 13041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:13.024767 13041 hybrid_clock.cc:648] HybridClock initialized: now 1786323433024694 us; error 44 us; skew 500 ppm
I20260810 00:57:13.025058 13041 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:13 dist-test-slave-jlzn krb5kdc[13019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323433, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:13.031138 13041 init.cc:377] Logged in from keytab as kudu/127.10.97.126@KRBTEST.COM (short username kudu)
I20260810 00:57:13.032613 13041 webserver.cc:460] Webserver started at http://127.10.97.126:42571/ using document root <none> and password file <none>
I20260810 00:57:13.033253 13041 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:13.033356 13041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:13.033659 13041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:13.036545 13041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-0/data/instance:
uuid: "ea8132f47f0a489fb3b1402a0d89d27b"
format_stamp: "Formatted at 2026-08-10 00:57:13 on dist-test-slave-jlzn"
server_key: "774d984bd4c4c96dde45334d171842d9"
server_key_iv: "3b4ab25680703a69291f09df7bdf80bc"
server_key_version: "encryptionkey@0"
I20260810 00:57:13.037288 13041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-0/wal/instance:
uuid: "ea8132f47f0a489fb3b1402a0d89d27b"
format_stamp: "Formatted at 2026-08-10 00:57:13 on dist-test-slave-jlzn"
server_key: "774d984bd4c4c96dde45334d171842d9"
server_key_iv: "3b4ab25680703a69291f09df7bdf80bc"
server_key_version: "encryptionkey@0"
I20260810 00:57:13.042078 13041 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260810 00:57:13.045252 13066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:13.046422 13041 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260810 00:57:13.046613 13041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-0/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-0/wal
uuid: "ea8132f47f0a489fb3b1402a0d89d27b"
format_stamp: "Formatted at 2026-08-10 00:57:13 on dist-test-slave-jlzn"
server_key: "774d984bd4c4c96dde45334d171842d9"
server_key_iv: "3b4ab25680703a69291f09df7bdf80bc"
server_key_version: "encryptionkey@0"
I20260810 00:57:13.046813 13041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:13.073220 13041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:13.077121 13041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:13.077359 13041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:13.094187 13041 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.126:33355
I20260810 00:57:13.094206 13146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.126:33355 every 8 connection(s)
I20260810 00:57:13.095438 13041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-0/data/info.pb
I20260810 00:57:13.098830 13148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:13.099900 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 13041
I20260810 00:57:13.100085 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-0/wal/instance
I20260810 00:57:13.100515 10629 external_mini_cluster.cc:1428] Setting key 5d67b261feeee347f46f19673d3268f3
I20260810 00:57:13.104987 13148 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:57:13 dist-test-slave-jlzn krb5kdc[13019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323433, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:57:13.125555 13151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.107716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43520 (local address 127.10.97.126:33355)
0810 00:57:13.109211 (+ 1495us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:13.109219 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:13.109274 (+ 55us) server_negotiation.cc:407] Connection header received
0810 00:57:13.110704 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:13.110711 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:13.110968 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:13.111167 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:13.112260 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:13.113867 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:13.115171 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:13.115689 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:13.115863 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:13.119325 (+ 3462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:13.119333 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:13.119342 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:13.119375 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:13.121895 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:13.122703 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:13.122709 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:13.122711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:13.122852 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:13.123239 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:13.123242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:13.123244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:13.123669 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:13.123854 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:13.124385 (+ 531us) server_negotiation.cc:299] Negotiation successful
0810 00:57:13.124622 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1301,"spinlock_wait_cycles":11520,"thread_start_us":1094,"threads_started":1}
I20260810 00:57:13.125600 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.107717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33355 (local address 127.0.0.1:43520)
0810 00:57:13.108560 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.108618 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:13.110352 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:13.111167 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:13.111172 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:13.111706 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:13.112059 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:13.112076 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:13.114001 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:13.114005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:13.115029 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:13.115045 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:13.115921 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:13.115929 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:13.116101 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:13.116667 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:13.116699 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:13.119117 (+ 2418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:13.122017 (+ 2900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:13.122029 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:13.122039 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:13.122578 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:13.122955 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:13.122960 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:13.122963 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:13.123150 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:13.123825 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:13.123832 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:13.124033 (+ 201us) client_negotiation.cc:769] Sending connection context
0810 00:57:13.124265 (+ 232us) client_negotiation.cc:240] Negotiation successful
0810 00:57:13.124837 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"mutex_wait_us":127,"thread_start_us":323,"threads_started":1}
I20260810 00:57:13.128026 13148 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:57:13.128979 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.128631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:40863 (local address 127.0.0.1:58800)
0810 00:57:13.128782 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.128880 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 00:57:13.129258 13070 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.97.125:40863: connect: Connection refused (error 111)
I20260810 00:57:13.130697 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.130299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:40863 (local address 127.0.0.1:58808)
0810 00:57:13.130458 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.130550 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 00:57:13.130908 13148 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.125:40863: Network error: Client connection negotiation failed: client connection to 127.10.97.125:40863: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.10.97.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.125.keytab.
Entry for principal kudu/127.10.97.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.125.keytab.
I20260810 00:57:13.165180 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.125:40863
--webserver_interface=127.10.97.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.125.keytab
--principal=kudu/127.10.97.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:33975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:33355,127.10.97.125:40863,127.10.97.124:39411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:57:13.180258 13148 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } attempt: 1
I20260810 00:57:13.181470 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.180935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:40863 (local address 127.0.0.1:58818)
0810 00:57:13.181167 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.181307 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 00:57:13.182476 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.182059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:40863 (local address 127.0.0.1:58822)
0810 00:57:13.182233 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.182326 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 00:57:13.182717 13148 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.125:40863: Network error: Client connection negotiation failed: client connection to 127.10.97.125:40863: connect: Connection refused (error 111)
I20260810 00:57:13.250972 13148 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } attempt: 2
I20260810 00:57:13.252267 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.251697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:40863 (local address 127.0.0.1:58830)
0810 00:57:13.251968 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.252084 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 00:57:13.253295 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.252889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:40863 (local address 127.0.0.1:58832)
0810 00:57:13.253036 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.253138 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 00:57:13.253527 13148 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.125:40863: Network error: Client connection negotiation failed: client connection to 127.10.97.125:40863: connect: Connection refused (error 111)
I20260810 00:57:13.344790 13148 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } attempt: 3
I20260810 00:57:13.345945 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.345451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:40863 (local address 127.0.0.1:58840)
0810 00:57:13.345669 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.345781 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 00:57:13.347004 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.346604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:40863 (local address 127.0.0.1:58846)
0810 00:57:13.346755 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.346841 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 00:57:13.347246 13148 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.125:40863: Network error: Client connection negotiation failed: client connection to 127.10.97.125:40863: connect: Connection refused (error 111)
W20260810 00:57:13.480960 13156 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 00:57:13.481249 13156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:13.481320 13156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:13.481400 13156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:13.488368 13156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:57:13.488445 13156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:13.488508 13156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:13.488556 13156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:57:13.488615 13156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:57:13.490509 13148 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } attempt: 4
I20260810 00:57:13.491765 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.491121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:40863 (local address 127.0.0.1:58858)
0810 00:57:13.491467 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.491592 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
I20260810 00:57:13.492808 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.492398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:40863 (local address 127.0.0.1:58866)
0810 00:57:13.492551 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.492659 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.125:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 00:57:13.493010 13148 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.125:40863: Network error: Client connection negotiation failed: client connection to 127.10.97.125:40863: connect: Connection refused (error 111)
I20260810 00:57:13.498374 13156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:33355,127.10.97.125:40863,127.10.97.124:39411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.125.keytab
--principal=kudu/127.10.97.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.125:40863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.97.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:13.499408 13156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:13.500626 13156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:13.511375 13162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:13.511445 13161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:13.511435 13165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:13.512127 13156 server_base.cc:1034] running on GCE node
I20260810 00:57:13.512814 13156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:13.514020 13156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:13.515236 13156 hybrid_clock.cc:648] HybridClock initialized: now 1786323433515167 us; error 33 us; skew 500 ppm
I20260810 00:57:13.515520 13156 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:13 dist-test-slave-jlzn krb5kdc[13019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323433, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:13.518509 13156 init.cc:377] Logged in from keytab as kudu/127.10.97.125@KRBTEST.COM (short username kudu)
I20260810 00:57:13.519858 13156 webserver.cc:460] Webserver started at http://127.10.97.125:41211/ using document root <none> and password file <none>
I20260810 00:57:13.520474 13156 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:13.520567 13156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:13.520855 13156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:13.523543 13156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-1/data/instance:
uuid: "a8a35bb8bc1e43c09af43be443d3fb40"
format_stamp: "Formatted at 2026-08-10 00:57:13 on dist-test-slave-jlzn"
server_key: "989d72b48c93f7fab856f83e38ee4b5a"
server_key_iv: "4c3b37e151138906fc62497960781f24"
server_key_version: "encryptionkey@0"
I20260810 00:57:13.524317 13156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-1/wal/instance:
uuid: "a8a35bb8bc1e43c09af43be443d3fb40"
format_stamp: "Formatted at 2026-08-10 00:57:13 on dist-test-slave-jlzn"
server_key: "989d72b48c93f7fab856f83e38ee4b5a"
server_key_iv: "4c3b37e151138906fc62497960781f24"
server_key_version: "encryptionkey@0"
I20260810 00:57:13.529012 13156 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 00:57:13.532110 13176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:13.533274 13156 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:57:13.533425 13156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-1/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-1/wal
uuid: "a8a35bb8bc1e43c09af43be443d3fb40"
format_stamp: "Formatted at 2026-08-10 00:57:13 on dist-test-slave-jlzn"
server_key: "989d72b48c93f7fab856f83e38ee4b5a"
server_key_iv: "4c3b37e151138906fc62497960781f24"
server_key_version: "encryptionkey@0"
I20260810 00:57:13.533605 13156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:13.581928 13156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:13.585872 13156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:13.586100 13156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:13.602734 13156 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.125:40863
I20260810 00:57:13.602753 13258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.125:40863 every 8 connection(s)
I20260810 00:57:13.603931 13156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-1/data/info.pb
I20260810 00:57:13.607226 13259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:13.608937 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 13156
I20260810 00:57:13.609128 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-1/wal/instance
I20260810 00:57:13.609665 10629 external_mini_cluster.cc:1428] Setting key b2b7589ea6b9ddd0927cd21412c46170
I20260810 00:57:13.615078 13259 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:57:13 dist-test-slave-jlzn krb5kdc[13019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323433, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.125@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:57:13.634680 13151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.617773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43534 (local address 127.10.97.126:33355)
0810 00:57:13.617966 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:13.617973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:13.619083 (+ 1110us) server_negotiation.cc:407] Connection header received
0810 00:57:13.620896 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:13.620901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:13.620983 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:13.621093 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:13.622377 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:13.623262 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:13.624701 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:13.625142 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:13.625256 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:13.628793 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:13.628799 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:13.628803 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:13.628841 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:13.631895 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:13.632545 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:13.632550 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:13.632553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:13.632721 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:13.633088 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:13.633093 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:13.633096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:13.633431 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:13.633574 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:13.634176 (+ 602us) server_negotiation.cc:299] Negotiation successful
0810 00:57:13.634384 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 00:57:13.635289 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.618196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33355 (local address 127.0.0.1:43534)
0810 00:57:13.618917 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.618970 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:13.620647 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:13.621251 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:13.621259 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:13.621731 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:13.622215 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:13.622231 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:13.623388 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:13.623391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:13.624556 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:13.624568 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:13.625349 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:13.625357 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:13.625615 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:13.626212 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:13.626239 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:13.628557 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:13.632011 (+ 3454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:13.632018 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:13.632028 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:13.632415 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:13.632822 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:13.632825 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:13.632827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:13.632976 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:13.633547 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:13.633552 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:13.633822 (+ 270us) client_negotiation.cc:769] Sending connection context
0810 00:57:13.634025 (+ 203us) client_negotiation.cc:240] Negotiation successful
0810 00:57:13.634343 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":306,"threads_started":1}
I20260810 00:57:13.636775 13259 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.97.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.97.124@KRBTEST.COM" created.
Aug 10 00:57:13 dist-test-slave-jlzn krb5kdc[13019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323433, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.125@KRBTEST.COM for kudu/127.10.97.125@KRBTEST.COM
I20260810 00:57:13.652711 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.637495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:40863 (local address 127.0.0.1:58872)
0810 00:57:13.637683 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.637701 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:13.637802 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:13.638814 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:13.638818 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:13.639187 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:13.639315 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:13.639322 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:13.641214 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:13.641217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:13.642016 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:13.642026 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:13.643075 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:13.643079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:13.643219 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:13.643699 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:13.643715 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:13.646942 (+ 3227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:13.650211 (+ 3269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:13.650215 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:13.650219 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:13.650677 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:13.651020 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:13.651024 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:13.651027 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:13.651203 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:13.651866 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:13.651870 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:13.651997 (+ 127us) client_negotiation.cc:769] Sending connection context
0810 00:57:13.652129 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 00:57:13.652377 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":3}
I20260810 00:57:13.652810 13268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.637663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58872 (local address 127.10.97.125:40863)
0810 00:57:13.638351 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:13.638358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:13.638385 (+ 27us) server_negotiation.cc:407] Connection header received
0810 00:57:13.638436 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:13.638443 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:13.638658 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:13.638797 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:13.639481 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:13.641094 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:13.642155 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:13.642951 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:13.643071 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:13.647157 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:13.647169 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:13.647182 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:13.647213 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:13.650078 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:13.650789 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:13.650796 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:13.650799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:13.650919 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:13.651313 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:13.651317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:13.651320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:13.651716 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:13.651900 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:13.652273 (+ 373us) server_negotiation.cc:299] Negotiation successful
0810 00:57:13.652480 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":530,"thread_start_us":378,"threads_started":1}
I20260810 00:57:13.655181 13259 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:57:13.656209 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.655838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59802)
0810 00:57:13.655973 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.656076 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 00:57:13.656464 13180 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
I20260810 00:57:13.657934 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.657614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59804)
0810 00:57:13.657777 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.657846 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 00:57:13.658130 13259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:39411: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.97.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.124.keytab.
Entry for principal kudu/127.10.97.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.124.keytab.
I20260810 00:57:13.676900 10629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskH92XUb/build/asan/bin/kudu
/tmp/dist-test-taskH92XUb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.124:39411
--webserver_interface=127.10.97.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.124.keytab
--principal=kudu/127.10.97.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.97.84:33975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.97.126:33355,127.10.97.125:40863,127.10.97.124:39411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 00:57:13.707358 13259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } attempt: 1
I20260810 00:57:13.708355 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.707943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59814)
0810 00:57:13.708113 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.708216 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 00:57:13.709332 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.709009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59820)
0810 00:57:13.709160 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.709233 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 00:57:13.709518 13259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:39411: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
I20260810 00:57:13.777710 13259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } attempt: 2
I20260810 00:57:13.778800 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.778383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59828)
0810 00:57:13.778562 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.778650 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 00:57:13.779748 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.779437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59832)
0810 00:57:13.779565 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.779632 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 00:57:13.780036 13259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:39411: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
I20260810 00:57:13.792238 13148 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } attempt: 5
Aug 10 00:57:13 dist-test-slave-jlzn krb5kdc[13019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323433, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for kudu/127.10.97.125@KRBTEST.COM
I20260810 00:57:13.804481 13268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.792877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58880 (local address 127.10.97.125:40863)
0810 00:57:13.793035 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:13.793040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:13.793108 (+ 68us) server_negotiation.cc:407] Connection header received
0810 00:57:13.793308 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:13.793312 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:13.793373 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:13.793457 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:13.794241 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:13.794925 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:13.796135 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:13.796579 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:13.796691 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:13.799621 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:13.799627 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:13.799631 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:13.799662 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:13.802089 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:13.802689 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:13.802695 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:13.802698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:13.802819 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:13.803173 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:13.803177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:13.803179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:13.803445 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:13.803575 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:13.803977 (+ 402us) server_negotiation.cc:299] Negotiation successful
0810 00:57:13.804188 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 00:57:13.804481 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.792826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:40863 (local address 127.0.0.1:58880)
0810 00:57:13.793015 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.793033 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:13.793176 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:13.793511 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:13.793516 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:13.793909 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:13.794091 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:13.794100 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:13.795053 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:13.795059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:13.795977 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:13.795991 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:13.796722 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:13.796728 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:13.796938 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:13.797502 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:13.797519 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:13.799448 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:13.802221 (+ 2773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:13.802226 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:13.802231 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:13.802580 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:13.802921 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:13.802926 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:13.802930 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:13.803064 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:13.803558 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:13.803562 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:13.803715 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 00:57:13.803868 (+ 153us) client_negotiation.cc:240] Negotiation successful
0810 00:57:13.804130 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 00:57:13.805717 13148 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:57:13.806607 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.806255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59834)
0810 00:57:13.806402 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.806469 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 00:57:13.807842 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.807505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59844)
0810 00:57:13.807639 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.807719 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 00:57:13.808035 13148 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:39411: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
I20260810 00:57:13.859227 13148 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } attempt: 1
I20260810 00:57:13.860291 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.859871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59850)
0810 00:57:13.860039 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.860134 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 00:57:13.861227 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.860901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59860)
0810 00:57:13.861047 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.861114 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 00:57:13.861428 13148 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:39411: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
I20260810 00:57:13.871284 13259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } attempt: 3
I20260810 00:57:13.872241 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.871862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59872)
0810 00:57:13.872018 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.872093 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 00:57:13.873277 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.872863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59880)
0810 00:57:13.873084 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.873172 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
W20260810 00:57:13.873478 13259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:39411: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
I20260810 00:57:13.929610 13148 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } attempt: 2
I20260810 00:57:13.930639 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.930183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59894)
0810 00:57:13.930394 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.930478 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 00:57:13.931727 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:13.931378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59900)
0810 00:57:13.931548 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:13.931620 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 00:57:13.931942 13148 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:39411: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
W20260810 00:57:14.000463 13272 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 00:57:14.000738 13272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 00:57:14.000805 13272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 00:57:14.000880 13272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 00:57:14.007614 13272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 00:57:14.007688 13272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 00:57:14.007750 13272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 00:57:14.007814 13272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 00:57:14.007890 13272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 00:57:14.016672 13259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } attempt: 4
I20260810 00:57:14.017122 13272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.97.84:33975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.97.126:33355,127.10.97.125:40863,127.10.97.124:39411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/krb5kdc/kudu_127.10.97.124.keytab
--principal=kudu/127.10.97.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.97.124:39411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.97.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 23:54:02 UTC on ccycloud.quasar-sxogyf.root.comops.site
build id 4576868
ASAN enabled
I20260810 00:57:14.017788 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.017319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59914)
0810 00:57:14.017548 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:14.017634 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 00:57:14.018362 13272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:57:14.018687 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.018363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59916)
0810 00:57:14.018504 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:14.018573 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 00:57:14.018896 13259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:39411: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
I20260810 00:57:14.019665 13272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:57:14.030714 13279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:14.030710 13280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:57:14.031122 13284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:57:14.031498 13272 server_base.cc:1034] running on GCE node
I20260810 00:57:14.032264 13272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 00:57:14.033473 13272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 00:57:14.034695 13272 hybrid_clock.cc:648] HybridClock initialized: now 1786323434034622 us; error 43 us; skew 500 ppm
I20260810 00:57:14.035010 13272 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 00:57:14 dist-test-slave-jlzn krb5kdc[13019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323434, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 00:57:14.038120 13272 init.cc:377] Logged in from keytab as kudu/127.10.97.124@KRBTEST.COM (short username kudu)
I20260810 00:57:14.038122 13148 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } attempt: 3
I20260810 00:57:14.039089 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.038682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59928)
0810 00:57:14.038874 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:14.038951 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 00:57:14.039603 13272 webserver.cc:460] Webserver started at http://127.10.97.124:39033/ using document root <none> and password file <none>
I20260810 00:57:14.040184 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.039710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59938)
0810 00:57:14.039959 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:14.040046 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260810 00:57:14.040367 13148 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.97.124:39411: Network error: Client connection negotiation failed: client connection to 127.10.97.124:39411: connect: Connection refused (error 111)
I20260810 00:57:14.040578 13272 fs_manager.cc:359] Metadata directory not provided
I20260810 00:57:14.040683 13272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:57:14.040974 13272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:57:14.043666 13272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-2/data/instance:
uuid: "c734015b3f1049f1984b86535138bcb7"
format_stamp: "Formatted at 2026-08-10 00:57:14 on dist-test-slave-jlzn"
server_key: "446492d602eb21aae7c8da49d1e91ace"
server_key_iv: "e9f85c4f6318cd71c25f72d4dae7d47a"
server_key_version: "encryptionkey@0"
I20260810 00:57:14.044449 13272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-2/wal/instance:
uuid: "c734015b3f1049f1984b86535138bcb7"
format_stamp: "Formatted at 2026-08-10 00:57:14 on dist-test-slave-jlzn"
server_key: "446492d602eb21aae7c8da49d1e91ace"
server_key_iv: "e9f85c4f6318cd71c25f72d4dae7d47a"
server_key_version: "encryptionkey@0"
I20260810 00:57:14.049079 13272 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:57:14.052343 13295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:57:14.053474 13272 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:57:14.053628 13272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-2/data,/tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-2/wal
uuid: "c734015b3f1049f1984b86535138bcb7"
format_stamp: "Formatted at 2026-08-10 00:57:14 on dist-test-slave-jlzn"
server_key: "446492d602eb21aae7c8da49d1e91ace"
server_key_iv: "e9f85c4f6318cd71c25f72d4dae7d47a"
server_key_version: "encryptionkey@0"
I20260810 00:57:14.053812 13272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:57:14.076592 13272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:57:14.080415 13272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:57:14.080647 13272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:57:14.097810 13272 rpc_server.cc:307] RPC server started. Bound to: 127.10.97.124:39411
I20260810 00:57:14.097867 13368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.97.124:39411 every 8 connection(s)
I20260810 00:57:14.098980 13272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-2/data/info.pb
I20260810 00:57:14.100412 10629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskH92XUb/build/asan/bin/kudu as pid 13272
I20260810 00:57:14.100605 10629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskH92XUb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786323077328248-10629-0/minicluster-data/master-2/wal/instance
I20260810 00:57:14.101013 10629 external_mini_cluster.cc:1428] Setting key 6e4eb8fc28c10b80cde2f063fbc330e4
I20260810 00:57:14.103173 13369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:57:14.112056 13369 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:57:14 dist-test-slave-jlzn krb5kdc[13019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.124@KRBTEST.COM
Aug 10 00:57:14 dist-test-slave-jlzn krb5kdc[13019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323434, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.124@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
I20260810 00:57:14.123617 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.102672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59950 (local address 127.10.97.124:39411)
0810 00:57:14.103511 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.103526 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.103884 (+ 358us) server_negotiation.cc:407] Connection header received
0810 00:57:14.105982 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.106017 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.106489 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.106897 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.108230 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.110275 (+ 2045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.111511 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.112515 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.112706 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.116663 (+ 3957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:14.116673 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:14.116684 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:14.116716 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:14.119869 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.120508 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.120516 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.120524 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.120744 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.121108 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.121112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.121114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.121637 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:14.121950 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:14.122290 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 00:57:14.122660 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"spinlock_wait_cycles":17664,"thread_start_us":382,"threads_started":1}
I20260810 00:57:14.128762 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.114495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.126:33355 (local address 127.0.0.1:43550)
0810 00:57:14.115153 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:14.115196 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:14.115337 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:14.115705 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:14.115711 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:14.116291 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:14.116527 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:14.116538 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.117526 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.117532 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:14.119146 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:14.119160 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.119906 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.119913 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:14.120077 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.120568 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:14.120589 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:14.122838 (+ 2249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:14.126120 (+ 3282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:14.126133 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:14.126139 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:14.126711 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:14.127119 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:14.127123 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:14.127126 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:14.127288 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:14.127925 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:14.127931 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:14.128069 (+ 138us) client_negotiation.cc:769] Sending connection context
0810 00:57:14.128204 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 00:57:14.128392 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":321,"threads_started":1}
I20260810 00:57:14.128819 13151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.114645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43550 (local address 127.10.97.126:33355)
0810 00:57:14.114833 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.114839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.115288 (+ 449us) server_negotiation.cc:407] Connection header received
0810 00:57:14.115454 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.115460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.115538 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.115641 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.116679 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.117373 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.119288 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.119710 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.119831 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.123011 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:14.123016 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:14.123019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:14.123049 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:14.125987 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.126877 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.126881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.126883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.127018 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.127405 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.127410 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.127413 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.127742 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:14.127943 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:14.128329 (+ 386us) server_negotiation.cc:299] Negotiation successful
0810 00:57:14.128533 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 00:57:14.130223 13369 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:57:14 dist-test-slave-jlzn krb5kdc[13019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.126@KRBTEST.COM
Aug 10 00:57:14 dist-test-slave-jlzn krb5kdc[13019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323434, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.124@KRBTEST.COM for kudu/127.10.97.125@KRBTEST.COM
I20260810 00:57:14.140954 13377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.126426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43560 (local address 127.10.97.126:33355)
0810 00:57:14.126979 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.126983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.127000 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:57:14.127050 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.127053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.127113 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.127219 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.128588 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.129768 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.131113 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.131653 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.131772 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.135934 (+ 4162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:14.135938 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:14.135941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:14.135968 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:14.138457 (+ 2489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.139196 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.139200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.139203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.139332 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.139737 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.139740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.139742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.140041 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:14.140249 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:14.140537 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 00:57:14.140679 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":240,"threads_started":1}
I20260810 00:57:14.143286 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.130747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.125:40863 (local address 127.0.0.1:58886)
0810 00:57:14.130884 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:14.130900 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:14.130974 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:14.131312 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:14.131316 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:14.131742 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:14.131930 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:14.131941 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.133164 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.133167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:14.133982 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:14.133992 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.134765 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.134770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:14.134946 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.135407 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:14.135429 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:14.137517 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:14.140936 (+ 3419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:14.140941 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:14.140945 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:14.141393 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:14.141820 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:14.141824 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:14.141827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:14.141984 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:14.142534 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:14.142539 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:14.142708 (+ 169us) client_negotiation.cc:769] Sending connection context
0810 00:57:14.142890 (+ 182us) client_negotiation.cc:240] Negotiation successful
0810 00:57:14.143065 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 00:57:14.143633 13268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.130860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58886 (local address 127.10.97.125:40863)
0810 00:57:14.131049 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.131055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.131072 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:57:14.131129 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.131132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.131198 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.131271 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.132071 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.133031 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.134123 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.134620 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.134712 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.137681 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:14.137686 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:14.137689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:14.137718 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:14.140802 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.141528 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.141534 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.141537 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.141713 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.142084 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.142088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.142092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.142399 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:14.142535 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:14.143017 (+ 482us) server_negotiation.cc:299] Negotiation successful
0810 00:57:14.143285 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 00:57:14.144481 13369 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 00:57:14 dist-test-slave-jlzn krb5kdc[13019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.97.125@KRBTEST.COM
Aug 10 00:57:14 dist-test-slave-jlzn krb5kdc[13019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323434, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.124@KRBTEST.COM for kudu/127.10.97.124@KRBTEST.COM
I20260810 00:57:14.156450 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.142520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58898 (local address 127.10.97.125:40863)
0810 00:57:14.143019 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.143024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.143044 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:57:14.143084 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.143088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.143146 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.143249 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.144538 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.145410 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.146830 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.147230 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.147337 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.151406 (+ 4069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:14.151413 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:14.151418 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:14.151453 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:14.153896 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.154528 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.154533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.154536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.154694 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.155072 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.155077 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.155080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.155351 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:14.155556 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:14.155873 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 00:57:14.156084 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":217,"threads_started":1}
I20260810 00:57:14.156535 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.145123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59958 (local address 127.10.97.124:39411)
0810 00:57:14.145272 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.145277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.145379 (+ 102us) server_negotiation.cc:407] Connection header received
0810 00:57:14.145523 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.145527 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.145593 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.145697 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.146288 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.147127 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.148244 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.148631 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.148729 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.151669 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:14.151673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:14.151676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:14.151703 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:14.154065 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.154660 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.154664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.154666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.154805 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.155156 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.155160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.155162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.155392 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:14.155527 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:14.155936 (+ 409us) server_negotiation.cc:299] Negotiation successful
0810 00:57:14.156270 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":61}
I20260810 00:57:14.156629 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.145167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59958)
0810 00:57:14.145314 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:14.145330 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:14.145432 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:14.145740 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:14.145743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:14.146045 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:14.146161 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:14.146168 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.147238 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.147243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:14.148112 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:14.148130 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.148781 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.148787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:14.148948 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.149406 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:14.149421 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:14.151506 (+ 2085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:14.154197 (+ 2691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:14.154201 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:14.154203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:14.154550 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:14.154919 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:14.154922 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:14.154924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:14.155055 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:14.155515 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:14.155520 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:14.155657 (+ 137us) client_negotiation.cc:769] Sending connection context
0810 00:57:14.155821 (+ 164us) client_negotiation.cc:240] Negotiation successful
0810 00:57:14.156083 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 00:57:14.160514 10629 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 00:57:14.161291 13369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7: Bootstrap starting.
I20260810 00:57:14.164144 13369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:14.165094 13369 log.cc:824] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:14.162676 13108 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ea8132f47f0a489fb3b1402a0d89d27b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43560
I20260810 00:57:14.167526 13369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7: No bootstrap required, opened a new log
I20260810 00:57:14.174237 13369 raft_consensus.cc:348] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } } peers { permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } } peers { permanent_uuid: "c734015b3f1049f1984b86535138bcb7" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } }
I20260810 00:57:14.174523 13369 raft_consensus.cc:729] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c734015b3f1049f1984b86535138bcb7, State: Initialized, Role: FOLLOWER
I20260810 00:57:14.174901 13369 consensus_queue.cc:260] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } } peers { permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } } peers { permanent_uuid: "c734015b3f1049f1984b86535138bcb7" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } }
I20260810 00:57:14.175652 13381 sys_catalog.cc:455] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } } peers { permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } } peers { permanent_uuid: "c734015b3f1049f1984b86535138bcb7" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } } }
I20260810 00:57:14.175984 13381 sys_catalog.cc:458] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:57:14.176795 13369 sys_catalog.cc:564] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 00:57:14.185468 13399 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 00:57:14.185600 13399 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 00:57:14.183197 13272 master_runner.cc:186] Error getting master registration for 127.10.97.126:33355: OK, instance_id { permanent_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" instance_seqno: 1786323433089400 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 00:57:14.193807 13272 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 00:57:14.217571 13148 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } attempt: 4
Aug 10 00:57:14 dist-test-slave-jlzn krb5kdc[13019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323433, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.126@KRBTEST.COM for kudu/127.10.97.124@KRBTEST.COM
I20260810 00:57:14.232102 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.218165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59966)
0810 00:57:14.218332 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:14.218350 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:14.218441 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:14.218768 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:14.218773 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:14.219161 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:14.219321 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:14.219331 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.220367 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.220373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:14.221332 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:14.221344 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.222046 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.222052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:14.222251 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.222869 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:14.222891 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:14.225653 (+ 2762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:14.229421 (+ 3768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:14.229426 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:14.229430 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:14.229920 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:14.230274 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:14.230279 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:14.230283 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:14.230463 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:14.231086 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:14.231091 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:14.231249 (+ 158us) client_negotiation.cc:769] Sending connection context
0810 00:57:14.231401 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 00:57:14.231697 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 00:57:14.232120 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.218253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59966 (local address 127.10.97.124:39411)
0810 00:57:14.218420 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.218426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.218449 (+ 23us) server_negotiation.cc:407] Connection header received
0810 00:57:14.218565 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.218571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.218631 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.218713 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.219461 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.220241 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.221469 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.221910 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.222017 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.225835 (+ 3818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:14.225841 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:14.225846 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:14.225882 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:14.229289 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.230028 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.230033 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.230037 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.230170 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.230578 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.230582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.230586 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.230955 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:14.231091 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:14.231528 (+ 437us) server_negotiation.cc:299] Negotiation successful
0810 00:57:14.231740 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 00:57:14.238266 13148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b: Bootstrap starting.
I20260810 00:57:14.242177 13148 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:14.243403 13148 log.cc:824] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:14.246562 13148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b: No bootstrap required, opened a new log
I20260810 00:57:14.253868 13148 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } } peers { permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } } peers { permanent_uuid: "c734015b3f1049f1984b86535138bcb7" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } }
I20260810 00:57:14.254244 13148 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea8132f47f0a489fb3b1402a0d89d27b, State: Initialized, Role: FOLLOWER
I20260810 00:57:14.254721 13148 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } } peers { permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } } peers { permanent_uuid: "c734015b3f1049f1984b86535138bcb7" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } }
I20260810 00:57:14.255617 13405 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } } peers { permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } } peers { permanent_uuid: "c734015b3f1049f1984b86535138bcb7" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } } }
I20260810 00:57:14.256052 13405 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:57:14.256819 13148 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:57:14.256937 13108 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 00:57:14.257122 13108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:14.259725 13108 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } } peers { permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } } peers { permanent_uuid: "c734015b3f1049f1984b86535138bcb7" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } }
I20260810 00:57:14.261610 13330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c734015b3f1049f1984b86535138bcb7"
I20260810 00:57:14.262058 13330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:14.264008 13108 leader_election.cc:288] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [CANDIDATE]: Term 1 election: Requested vote from peers a8a35bb8bc1e43c09af43be443d3fb40 (127.10.97.125:40863), c734015b3f1049f1984b86535138bcb7 (127.10.97.124:39411)
I20260810 00:57:14.265484 13330 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea8132f47f0a489fb3b1402a0d89d27b in term 1.
I20260810 00:57:14.266075 13072 leader_election.cc:302] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c734015b3f1049f1984b86535138bcb7, ea8132f47f0a489fb3b1402a0d89d27b; no voters:
I20260810 00:57:14.266554 13405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:57:14.274016 13405 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [term 1 LEADER]: Becoming Leader. State: Replica: ea8132f47f0a489fb3b1402a0d89d27b, State: Running, Role: LEADER
I20260810 00:57:14.274549 13405 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } } peers { permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } } peers { permanent_uuid: "c734015b3f1049f1984b86535138bcb7" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } }
I20260810 00:57:14.280449 13412 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea8132f47f0a489fb3b1402a0d89d27b. Latest consensus state: current_term: 1 leader_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } } peers { permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } } peers { permanent_uuid: "c734015b3f1049f1984b86535138bcb7" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } } }
I20260810 00:57:14.280874 13412 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:14.288812 13417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:57:14.302805 13417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:57:14.318078 13259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } attempt: 5
I20260810 00:57:14.320672 13330 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7 [term 1 FOLLOWER]: Refusing update from remote peer ea8132f47f0a489fb3b1402a0d89d27b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:14.321611 13412 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c734015b3f1049f1984b86535138bcb7" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 10 00:57:14 dist-test-slave-jlzn krb5kdc[13019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786323433, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.97.125@KRBTEST.COM for kudu/127.10.97.124@KRBTEST.COM
W20260810 00:57:14.329797 13077 tablet.cc:2369] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 00:57:14.332376 13302 tablet.cc:2369] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:57:14.337457 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.319289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59972 (local address 127.10.97.124:39411)
0810 00:57:14.319457 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.319463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.319480 (+ 17us) server_negotiation.cc:407] Connection header received
0810 00:57:14.319534 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.319539 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.319642 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.319735 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.320904 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.322036 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.323797 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.324535 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.324659 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.328685 (+ 4026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:14.328692 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:14.328697 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:14.328734 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:14.333967 (+ 5233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.334784 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.334789 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.334793 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.334956 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.335379 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.335385 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.335388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.335714 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:14.335871 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:14.336888 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0810 00:57:14.337098 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 00:57:14.337888 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.318629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.97.124:39411 (local address 127.0.0.1:59972)
0810 00:57:14.318960 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0810 00:57:14.318975 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 00:57:14.319086 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 00:57:14.319956 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 00:57:14.319961 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 00:57:14.320465 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0810 00:57:14.320697 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:14.320709 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.322213 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.322220 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:14.323620 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 00:57:14.323645 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.324857 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.324863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 00:57:14.325073 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.325726 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 00:57:14.325750 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 00:57:14.328467 (+ 2717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 00:57:14.334119 (+ 5652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:14.334125 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:14.334129 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:14.334645 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:14.335083 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 00:57:14.335089 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 00:57:14.335093 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 00:57:14.335254 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 00:57:14.335935 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 00:57:14.335941 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 00:57:14.336097 (+ 156us) client_negotiation.cc:769] Sending connection context
0810 00:57:14.336239 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 00:57:14.336506 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232}
I20260810 00:57:14.343380 13259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40: Bootstrap starting.
I20260810 00:57:14.347070 13259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40: Neither blocks nor log segments found. Creating new log.
I20260810 00:57:14.348340 13259 log.cc:824] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40: Log is configured to *not* fsync() on all Append() calls
I20260810 00:57:14.357527 13259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40: No bootstrap required, opened a new log
I20260810 00:57:14.320284 13041 master_runner.cc:186] Error getting master registration for 127.10.97.125:40863: OK, instance_id { permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" instance_seqno: 1786323433598107 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 00:57:14.362062 13041 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 00:57:14.368178 13259 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } } peers { permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } } peers { permanent_uuid: "c734015b3f1049f1984b86535138bcb7" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } }
I20260810 00:57:14.368534 13259 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8a35bb8bc1e43c09af43be443d3fb40, State: Initialized, Role: FOLLOWER
I20260810 00:57:14.369001 13259 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } } peers { permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } } peers { permanent_uuid: "c734015b3f1049f1984b86535138bcb7" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } }
I20260810 00:57:14.377317 13259 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:57:14.377388 13221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a8a35bb8bc1e43c09af43be443d3fb40"
I20260810 00:57:14.378809 13221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:57:14.381999 13221 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea8132f47f0a489fb3b1402a0d89d27b in term 1.
I20260810 00:57:14.382517 13445 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } } peers { permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } } peers { permanent_uuid: "c734015b3f1049f1984b86535138bcb7" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } } }
I20260810 00:57:14.382959 13445 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:57:14.384033 13220 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40 [term 1 FOLLOWER]: Refusing update from remote peer ea8132f47f0a489fb3b1402a0d89d27b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:57:14.385689 13412 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:57:14.404544 13381 sys_catalog.cc:455] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea8132f47f0a489fb3b1402a0d89d27b. Latest consensus state: current_term: 1 leader_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } } peers { permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } } peers { permanent_uuid: "c734015b3f1049f1984b86535138bcb7" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } } }
I20260810 00:57:14.404860 13381 sys_catalog.cc:458] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:57:14.406699 13429 mvcc.cc:205] Tried to move back new op lower bound from 7316780786955771904 to 7316780786793820160. Current Snapshot: MvccSnapshot[applied={T|T < 7316780786955771904}]
I20260810 00:57:14.408516 13438 mvcc.cc:205] Tried to move back new op lower bound from 7316780786955771904 to 7316780786793820160. Current Snapshot: MvccSnapshot[applied={T|T < 7316780786955771904}]
I20260810 00:57:14.416639 13405 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } } peers { permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } } peers { permanent_uuid: "c734015b3f1049f1984b86535138bcb7" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } } }
I20260810 00:57:14.416922 13405 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:14.417558 13445 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea8132f47f0a489fb3b1402a0d89d27b. Latest consensus state: current_term: 1 leader_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } } peers { permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } } peers { permanent_uuid: "c734015b3f1049f1984b86535138bcb7" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } } }
I20260810 00:57:14.417901 13445 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:57:14.419767 13417 catalog_manager.cc:1347] Generated new cluster ID: f2abc572cc0c45bfb02b155c93c6edbb
I20260810 00:57:14.419879 13417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:57:14.420152 13381 sys_catalog.cc:455] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } } peers { permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } } peers { permanent_uuid: "c734015b3f1049f1984b86535138bcb7" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } } }
I20260810 00:57:14.420387 13381 sys_catalog.cc:458] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:57:14.434811 13412 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } } peers { permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } } peers { permanent_uuid: "c734015b3f1049f1984b86535138bcb7" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } } }
I20260810 00:57:14.435050 13412 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b [sys.catalog]: This master's current role is: LEADER
I20260810 00:57:14.438683 13450 mvcc.cc:205] Tried to move back new op lower bound from 7316780786955771904 to 7316780786793820160. Current Snapshot: MvccSnapshot[applied={T|T < 7316780786955771904}]
I20260810 00:57:14.439535 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.311594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58902 (local address 127.10.97.125:40863)
0810 00:57:14.311798 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.311820 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.378779 (+ 66959us) server_negotiation.cc:407] Connection header received
0810 00:57:14.379106 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.379112 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.379207 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.379278 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.381170 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.383349 (+ 2179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.404329 (+ 20980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.404994 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.405088 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.420025 (+ 14937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:14.420031 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:14.420035 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:14.420070 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:14.423454 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.429092 (+ 5638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.429097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.429101 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.429226 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.435842 (+ 6616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.435848 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.435852 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.436227 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:14.436464 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:14.438944 (+ 2480us) server_negotiation.cc:299] Negotiation successful
0810 00:57:14.439182 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 00:57:14.443852 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.335630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59982 (local address 127.10.97.124:39411)
0810 00:57:14.338558 (+ 2928us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.338563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.391061 (+ 52498us) server_negotiation.cc:407] Connection header received
0810 00:57:14.391300 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.391306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.391406 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.391476 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.393273 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.394343 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.405962 (+ 11619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.406662 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.406770 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.424263 (+ 17493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:14.424269 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:14.424274 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:14.424308 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:14.427992 (+ 3684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.436779 (+ 8787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.436785 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.436789 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.436944 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.437667 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.437673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.437677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.438056 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:14.438298 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:14.443194 (+ 4896us) server_negotiation.cc:299] Negotiation successful
0810 00:57:14.443436 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2814,"thread_start_us":50394,"threads_started":1}
I20260810 00:57:14.447180 13377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.303863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43562 (local address 127.10.97.126:33355)
0810 00:57:14.304082 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.304089 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.374276 (+ 70187us) server_negotiation.cc:407] Connection header received
0810 00:57:14.374737 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.374744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.374828 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.374897 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.400730 (+ 25833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.401770 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.409811 (+ 8041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.410544 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.410638 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.429658 (+ 19020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 00:57:14.429664 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 00:57:14.429669 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 00:57:14.429703 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 00:57:14.433104 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.440055 (+ 6951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.440062 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.440066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.440222 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 00:57:14.442072 (+ 1850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 00:57:14.442078 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 00:57:14.442081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 00:57:14.442428 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 00:57:14.442689 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:14.446583 (+ 3894us) server_negotiation.cc:299] Negotiation successful
0810 00:57:14.446833 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 00:57:14.451346 13445 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8132f47f0a489fb3b1402a0d89d27b" member_type: VOTER last_known_addr { host: "127.10.97.126" port: 33355 } } peers { permanent_uuid: "a8a35bb8bc1e43c09af43be443d3fb40" member_type: VOTER last_known_addr { host: "127.10.97.125" port: 40863 } } peers { permanent_uuid: "c734015b3f1049f1984b86535138bcb7" member_type: VOTER last_known_addr { host: "127.10.97.124" port: 39411 } } }
I20260810 00:57:14.451601 13445 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:57:14.459411 13417 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:57:14.460943 13417 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:57:14.484910 13417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea8132f47f0a489fb3b1402a0d89d27b: Generated new TSK 0
I20260810 00:57:14.485847 13417 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:57:14.502774 13473 catalog_manager.cc:1259] Loaded cluster ID: f2abc572cc0c45bfb02b155c93c6edbb
I20260810 00:57:14.502864 13473 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40: loading cluster ID for follower catalog manager: success
I20260810 00:57:14.506232 13473 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40: acquiring CA information for follower catalog manager: success
I20260810 00:57:14.508409 13473 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a8a35bb8bc1e43c09af43be443d3fb40: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 00:57:14.654850 13377 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:14.622506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43574 (local address 127.10.97.126:33355)
0810 00:57:14.622704 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.622711 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.652940 (+ 30229us) server_negotiation.cc:407] Connection header received
0810 00:57:14.653229 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.653237 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.653322 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.653395 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.654582 (+ 1187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43574: BlockingRecv error: recv got EOF from 127.0.0.1:43574 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260810 00:57:14.657397 13379 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:14.629330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58904 (local address 127.10.97.125:40863)
0810 00:57:14.629494 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.629500 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.655598 (+ 26098us) server_negotiation.cc:407] Connection header received
0810 00:57:14.655843 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.655849 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.655974 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.656044 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.657141 (+ 1097us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58904: BlockingRecv error: recv got EOF from 127.0.0.1:58904 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260810 00:57:14.659706 13374 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:14.639043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59986 (local address 127.10.97.124:39411)
0810 00:57:14.639213 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.639218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.658209 (+ 18991us) server_negotiation.cc:407] Connection header received
0810 00:57:14.658429 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.658435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.658510 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.658594 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.659446 (+ 852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59986: BlockingRecv error: recv got EOF from 127.0.0.1:59986 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 00:57:14.707150 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.691964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58906 (local address 127.10.97.125:40863)
0810 00:57:14.692141 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.692147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.699453 (+ 7306us) server_negotiation.cc:407] Connection header received
0810 00:57:14.699947 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.699953 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.700156 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.700291 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:14.700984 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.702154 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.704195 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.704872 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.704963 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.705651 (+ 688us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:14.706304 (+ 653us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:14.706423 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:14.706682 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 00:57:14.706766 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 00:57:14.724181 13374 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:14.722076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59996 (local address 127.10.97.124:39411)
0810 00:57:14.722276 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.722282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.722721 (+ 439us) server_negotiation.cc:407] Connection header received
0810 00:57:14.722888 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.722894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.723010 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.723121 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.723954 (+ 833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59996: BlockingRecv error: recv got EOF from 127.0.0.1:59996 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 00:57:14.738410 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.728200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58914 (local address 127.10.97.125:40863)
0810 00:57:14.728358 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.728363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.732309 (+ 3946us) server_negotiation.cc:407] Connection header received
0810 00:57:14.732474 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.732480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.732566 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.732663 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:14.733250 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.734384 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.736342 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.737027 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.737156 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.737667 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:14.737823 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:14.737910 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:14.738057 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 00:57:14.738131 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 00:57:14.747169 13374 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:14.745670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59998 (local address 127.10.97.124:39411)
0810 00:57:14.745832 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.745838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.745856 (+ 18us) server_negotiation.cc:407] Connection header received
0810 00:57:14.745916 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.745920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.746008 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.746107 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.746975 (+ 868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59998: BlockingRecv error: recv got EOF from 127.0.0.1:59998 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 00:57:14.764400 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.757853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58920 (local address 127.10.97.125:40863)
0810 00:57:14.758053 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.758058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.758595 (+ 537us) server_negotiation.cc:407] Connection header received
0810 00:57:14.758745 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.758750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.758828 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.758924 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:14.759335 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.760461 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.762363 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.763028 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.763156 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.763630 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:14.763792 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:14.763900 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:14.764032 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 00:57:14.764111 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260810 00:57:14.772630 13374 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:14.770579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60012 (local address 127.10.97.124:39411)
0810 00:57:14.770727 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.770732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.771196 (+ 464us) server_negotiation.cc:407] Connection header received
0810 00:57:14.771335 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.771340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.771465 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.771589 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.772400 (+ 811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60012: BlockingRecv error: recv got EOF from 127.0.0.1:60012 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 00:57:14.801620 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.795042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58930 (local address 127.10.97.125:40863)
0810 00:57:14.795215 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.795220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.795700 (+ 480us) server_negotiation.cc:407] Connection header received
0810 00:57:14.795888 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.795894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.795980 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.796095 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:14.796463 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.797584 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.799460 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.800181 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.800330 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.800808 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:14.800950 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:14.801069 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:14.801236 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 00:57:14.801315 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 00:57:14.810227 13374 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:14.808572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60026 (local address 127.10.97.124:39411)
0810 00:57:14.808823 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.808828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.808849 (+ 21us) server_negotiation.cc:407] Connection header received
0810 00:57:14.808915 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.808919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.809029 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.809135 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.810030 (+ 895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60026: BlockingRecv error: recv got EOF from 127.0.0.1:60026 (error 108)
Metrics: {"server-negotiator.queue_time_us":144}
I20260810 00:57:14.831892 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.821975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58932 (local address 127.10.97.125:40863)
0810 00:57:14.822149 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.822155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.824898 (+ 2743us) server_negotiation.cc:407] Connection header received
0810 00:57:14.825357 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.825363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.825541 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.825670 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:14.826478 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.827572 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.829674 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.830411 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.830522 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.831119 (+ 597us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:14.831235 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:14.831281 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:14.831462 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 00:57:14.831562 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260810 00:57:14.841645 13374 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:14.839898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60036 (local address 127.10.97.124:39411)
0810 00:57:14.840072 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.840078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.840251 (+ 173us) server_negotiation.cc:407] Connection header received
0810 00:57:14.840436 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.840441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.840523 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.840629 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.841429 (+ 800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60036: BlockingRecv error: recv got EOF from 127.0.0.1:60036 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 00:57:14.867606 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.860678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58942 (local address 127.10.97.125:40863)
0810 00:57:14.860833 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.860839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.861329 (+ 490us) server_negotiation.cc:407] Connection header received
0810 00:57:14.861472 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.861477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.861559 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.861653 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:14.862085 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.863256 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.865441 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.866130 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.866231 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.866814 (+ 583us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:14.866940 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:14.867003 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:14.867265 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 00:57:14.867344 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 00:57:14.883399 13374 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:14.881333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60052 (local address 127.10.97.124:39411)
0810 00:57:14.881492 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.881497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.881975 (+ 478us) server_negotiation.cc:407] Connection header received
0810 00:57:14.882142 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.882147 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.882238 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.882342 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.883178 (+ 836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60052: BlockingRecv error: recv got EOF from 127.0.0.1:60052 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 00:57:14.923523 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:14.918292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58952 (local address 127.10.97.125:40863)
0810 00:57:14.918486 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.918493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.918763 (+ 270us) server_negotiation.cc:407] Connection header received
0810 00:57:14.918951 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.918956 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.919025 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.919107 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:14.919547 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.920384 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.921847 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:14.922285 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:14.922406 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:14.922787 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:14.922927 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:14.923005 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:14.923162 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 00:57:14.923245 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260810 00:57:14.940136 13374 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:14.938273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60060 (local address 127.10.97.124:39411)
0810 00:57:14.938479 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:14.938484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:14.938816 (+ 332us) server_negotiation.cc:407] Connection header received
0810 00:57:14.938965 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:14.938968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:14.939050 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:14.939127 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:14.939900 (+ 773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60060: BlockingRecv error: recv got EOF from 127.0.0.1:60060 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 00:57:15.011961 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:15.007260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58968 (local address 127.10.97.125:40863)
0810 00:57:15.007454 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:15.007461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:15.007772 (+ 311us) server_negotiation.cc:407] Connection header received
0810 00:57:15.007990 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:15.007994 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:15.008079 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:15.008161 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:15.008581 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:15.009323 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:15.010590 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:15.010986 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:15.011084 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:15.011373 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:15.011480 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:15.011556 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:15.011643 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 00:57:15.011703 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 00:57:15.018306 13374 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:15.016790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60072 (local address 127.10.97.124:39411)
0810 00:57:15.016975 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:15.016980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:15.017216 (+ 236us) server_negotiation.cc:407] Connection header received
0810 00:57:15.017329 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:15.017333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:15.017400 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:15.017474 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:15.018118 (+ 644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60072: BlockingRecv error: recv got EOF from 127.0.0.1:60072 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 00:57:15.154224 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:15.149645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58970 (local address 127.10.97.125:40863)
0810 00:57:15.149820 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:15.149826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:15.150208 (+ 382us) server_negotiation.cc:407] Connection header received
0810 00:57:15.150396 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:15.150400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:15.150468 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:15.150537 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:15.150950 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:15.151665 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:15.152938 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:15.153330 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:15.153419 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:15.153726 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:15.153820 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:15.153878 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:15.153967 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 00:57:15.154024 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260810 00:57:15.161279 13374 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 00:57:15.159454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60082 (local address 127.10.97.124:39411)
0810 00:57:15.159620 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:15.159626 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:15.160017 (+ 391us) server_negotiation.cc:407] Connection header received
0810 00:57:15.160194 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:15.160200 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:15.160272 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:15.160353 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 00:57:15.161093 (+ 740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60082: BlockingRecv error: recv got EOF from 127.0.0.1:60082 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 00:57:15.187390 13399 catalog_manager.cc:1259] Loaded cluster ID: f2abc572cc0c45bfb02b155c93c6edbb
I20260810 00:57:15.187471 13399 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7: loading cluster ID for follower catalog manager: success
I20260810 00:57:15.189996 13399 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7: acquiring CA information for follower catalog manager: success
I20260810 00:57:15.191677 13399 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c734015b3f1049f1984b86535138bcb7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 00:57:15.425771 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:15.420882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58974 (local address 127.10.97.125:40863)
0810 00:57:15.421098 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:15.421105 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:15.421252 (+ 147us) server_negotiation.cc:407] Connection header received
0810 00:57:15.421456 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:15.421460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:15.421527 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:15.421607 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:15.422053 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:15.422783 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:15.424177 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:15.424575 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:15.424687 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:15.425069 (+ 382us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:15.425186 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:15.425262 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:15.425430 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 00:57:15.425523 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 00:57:15.440491 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:15.433451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60084 (local address 127.10.97.124:39411)
0810 00:57:15.434337 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:15.434343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:15.434363 (+ 20us) server_negotiation.cc:407] Connection header received
0810 00:57:15.434419 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:15.434422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:15.434596 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:15.434720 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:15.435266 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:15.436027 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:15.437875 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:15.438430 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:15.438546 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:15.439203 (+ 657us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:15.439922 (+ 719us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:15.440040 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:15.440214 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 00:57:15.440277 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752}
I20260810 00:57:15.458717 13644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:15.448476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43582 (local address 127.10.97.126:33355)
0810 00:57:15.449486 (+ 1010us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:15.449493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:15.449512 (+ 19us) server_negotiation.cc:407] Connection header received
0810 00:57:15.449623 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:15.449627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:15.449839 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:15.450016 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:15.451133 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:15.452352 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:15.455144 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:15.455962 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:15.456103 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:15.457277 (+ 1174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:15.457949 (+ 672us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:15.458077 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:15.458296 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 00:57:15.458387 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":897,"thread_start_us":813,"threads_started":1}
I20260810 00:57:15.464848 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 13041
I20260810 00:57:15.464898 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:15.448456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60094 (local address 127.10.97.124:39411)
0810 00:57:15.448725 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:15.448730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:15.458011 (+ 9281us) server_negotiation.cc:407] Connection header received
0810 00:57:15.458148 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:15.458153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:15.458233 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:15.458311 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:15.459397 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:15.460328 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:15.462069 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:15.462683 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:15.462798 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:15.463376 (+ 578us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:15.463485 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:15.463548 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:15.464575 (+ 1027us) server_negotiation.cc:299] Negotiation successful
0810 00:57:15.464660 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260810 00:57:15.465804 13141 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:15.466032 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 00:57:15.448430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58980 (local address 127.10.97.125:40863)
0810 00:57:15.448582 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 00:57:15.448588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 00:57:15.458610 (+ 10022us) server_negotiation.cc:407] Connection header received
0810 00:57:15.458753 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 00:57:15.458758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 00:57:15.458841 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 00:57:15.458914 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 00:57:15.460493 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:15.461480 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:15.463613 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 00:57:15.464194 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 00:57:15.464313 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 00:57:15.465165 (+ 852us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 00:57:15.465292 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 00:57:15.465375 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 00:57:15.465698 (+ 323us) server_negotiation.cc:299] Negotiation successful
0810 00:57:15.465772 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 00:57:15.578496 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 13041
I20260810 00:57:15.608893 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 13156
I20260810 00:57:15.609899 13253 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:15.713052 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 13156
I20260810 00:57:15.736218 10629 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskH92XUb/build/asan/bin/kudu pid 13272
I20260810 00:57:15.737282 13364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 00:57:15.841735 10629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskH92XUb/build/asan/bin/kudu with pid 13272
2026-08-10T00:57:15Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4638 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9697 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (358438 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST